Thematic response to a computer user&#39;s context, such as by a wearable personal computer

ABSTRACT

Techniques are disclosed for creating, modifying, categorizing, modeling, distributing, purchasing, selling, and otherwise using themes and theme-related information. Themes can represent various types of contextual aspects or situations, and can model high-level concepts of activities or states not reflected in individual contextual attributes that each model a single aspect of the state of a user, their computing device, the surrounding physical environment, and/or the current cyber-environment. Such themes specify inter-relationships among a set of contextual attributes, and can have associated theme-related information such as theme-specific attributes, theme layouts used to present information and functionality, CSes that provide theme attribute values, and CCs that process theme information. Disclosed techniques can identify one or more themes that currently match the modeled context, select one of the matching themes as a current theme, and provide an appropriate response (e.g., by presenting appropriate information and/or providing appropriate functionality) based on the current theme.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.09/825,159, filed Apr. 2, 2001 and currently pending. U.S. patentapplication Ser. No. 09/825,159 is a continuation-in-part of U.S. patentapplication Ser. No. 09/216,193, filed Dec. 18, 1998, now U.S. Pat. No.6,466,232 and entitled “Method and System for Controlling Presentationof Information to a User Based on the User's Condition”; of U.S. patentapplication Ser. No. 09/464,659, filed Dec. 15, 1999 and entitled“Storing and Recalling Information to Augment Human Memories”; and ofU.S. patent application Ser. No. 09/724,902, filed Nov. 28, 2000 andentitled “Dynamically Exchanging Computer User's Context,” which claimsthe benefit of provisional U.S. patent application Ser. No. 60/194,006filed Apr. 2, 2000. each of which are hereby incorporated by referencein their entirety.

U.S. patent application Ser. No. 09/825,159 also claims the benefit ofprovisional U.S. patent application Ser. No. 60/193,999, filed Apr. 2,2000 and entitled“Obtaining And Using Contextual Data For Selected TasksOr Scenarios, Such As For A Wearable Personal Computer,” and ofprovisional U.S. patent application Ser. No. 60/194,123, filed Apr. 2,2000 and entitled“Supplying And Consuming User Context Data,” both ofwhich are hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The invention described below relates generally to gathering and usingcontextual information, such as information about a user and the user'ssurroundings.

BACKGROUND

Existing computer systems provide little appreciation of a user'soverall condition or context, and as a result they can effectivelyrespond to only a limited number of changes in parameters that theymonitor. For example, with respect to the low-level physical status ofthe user, numerous devices exist for monitoring the physical parametersof the user, such as heart rate monitors that provide user pulse orheart rate data. While many of these devices simply provide informationto the user regarding current values of a user's health condition,others (e.g., a defibrillator or a system with an alarm) are capable ofproviding a corresponding response if a monitored parameter exceeds (orfalls below) a threshold value. However, since such devices lackimportant information about the specific context of the user (e.g.,whether the user is currently exercising or is currently sick), anyresponse will attempt to accommodate a wide range of user contexts andis thus unlikely to be optimal for the specific context of the user. Forexample, a defibrillator may provide too great or too small of aresuscitating charge simply because only one or a small number ofparameters of a person are being monitored.

In a similar manner, existing computer systems have little appreciationfor a user's current mental and emotional state, or for higher-levelabstractions of a user's physical activity (e.g., going jogging ordriving an automobile), and as a result are generally ineffective atanticipating tasks that a user is likely to perform or information thata user is likely to desire. In particular, since existing computersystems lack information about a user's current context, they cannotprovide information appropriate to that context or anticipate likelychanges in the context.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a user with an embodiment of a specializedcharacterization system operating, at least in part, on a wearablecomputer platform.

FIG. 2 is a block diagram of the specialized characterization system ofFIG. 1 having specialized context servers and clients.

FIG. 3 is a block diagram illustrating code modules, such as contextservers and clients, acting in conjunction with a characterizationmodule (CM).

FIG. 4 is a flow diagram illustrating basic steps for generatingattributes and providing output to a user based on the system of FIG. 3.

FIG. 5A is a flow diagram illustrating a context server for generatinglatitude, longitude and altitude attributes.

FIG. 5B is a data structure diagram illustrating fields or portions of alatitude attribute.

FIG. 6 is a flow diagram illustrating a context client for usinglatitude, longitude and altitude attributes.

FIG. 7 is a schematic diagram illustrating how attributes are derivedbased on input data sources.

FIG. 8 is a hierarchical list of attributes.

FIG. 9 is an alternative hierarchical list of attributes.

FIG. 10 is a block diagram illustrating an embodiment of a computingdevice suitable for using theme and theme layout information to presentappropriate information and functionality to a user based on the currentcontext.

FIGS. 11A–11L provide various examples of changing theme layoutpresentations based on changes to a current context.

FIGS. 11M–11O provide examples of a user interface for a user toexplicitly control various theme-related information.

FIGS. 12A–12H provide examples of a user interface for a user toexplicitly specify context information.

FIG. 13 provides an example of a theme data structure.

FIG. 14 is a block diagram illustrating an embodiment of a computingdevice suitable for distributing themes and theme-related information tovarious user computing devices.

FIG. 15 is a flow diagram of an embodiment of the Theme Usage routine.

FIG. 16 is a flow diagram of an embodiment of the Response Generatorsubroutine.

FIG. 17 is a flow diagram of an embodiment of the Theme Creator/Modifierroutine.

FIG. 18 is a flow diagram of an embodiment of the Theme Distributorroutine.

FIG. 19 is a flow diagram of an embodiment of the Theme Receiverroutine.

FIG. 20 is a flow diagram of an embodiment of the Automated ThemeCustomizer routine.

DETAILED DESCRIPTION

The following description provides specific details for a thoroughunderstanding of, and enabling description for, embodiments of theinvention. However, one skilled in the relevant art will understand thatthe invention may be practiced without these details. In otherinstances, well known structures and functions have not been shown ordescribed in detail to avoid unnecessarily obscuring embodiments of theinvention.

Techniques are described for creating, modifying, analyzing,characterizing, distributing, modeling, and using themes that representa context of a user. The themes each include related sets of attributesthat reflect the context of the user, including: (1) the user's mentalstate, emotional state, and physical or health condition; (2) the user'ssetting, situation or physical environment (including factors externalto the user that can be observed and/or manipulated by the user, such asthe state of the user's wearable computer); and (3) the user's logicaland data telecommunications environment (or “cyber-environment,”including information such as email addresses, nearby telecommunicationsaccess such as cell sites, wireless computer ports, etc.).

Such themes may be employed by various computing devices, although muchof the following description focuses on wearable computers as anexample. Wearable computers find enhanced benefit over fixed locationcomputers by often being more context aware or able to intelligentlyinterpret attributes, thereby more fully utilizing the capabilities ofthe wearable computer. Many computing applications, including thoserunning on wearable computers, act as context consumers or attributeclients in that they use such contextual attribute information todeliver context-aware functionality to end users. Many of theseapplications may use the same context information (i.e., the sameattribute or sets of attributes).

Described below is a general framework related to using context themes.In some embodiments, a theme may consolidate one or more context sourcesand context consumers specific to the theme. Modules that provideattributes are generally referred to as context servers (CS), whilemodules that process attributes are generally referred to as contextclients (CC). As described below, a characterization module (CM) acts asan attribute exchange mechanism for context information between such CSand CC applications or “code modules.” The terms context server, contextclient and characterization module, and CS, CC and CM are generally usedinterchangeably herein.

Each of the context servers receive data signals from input devices orother sources and then process such signals to produce contextinformation expressed within fundamental data structures called“attributes”. Attributes represent measures of specific context elementssuch as ambient temperature, latitude, and current user task. Eachattribute has a name and at least one value, and can additionally haveother properties such as uncertainty, units and timestamp. As generallyused herein, an “attribute” refers to a modeled aspect or element of theuser's condition or context. The terms “attribute” and “conditionvariable” are generally used interchangeably herein. Further details onattributes are described below.

In the general framework described in greater detail below, contextservers provide attribute values and/or themes to the CM, which in turnprovides some or all of such information to the context clients. Thisgeneral context framework encourages development of more abstractcontext information from simple data signals provided by input devices,because it provides a standard mechanism for the exchange of attributevalues provided by context servers. For example, simple contextinformation such as location signals provided by a global positioningsystem (GPS) receiver can be combined with other information such asambient noise signals and video input cues to answer more abstractcontext questions such as “How busy am I?” or “What is my currentactivity?”. In particular, GPS signals may indicate over time that auser is traveling at 30 m.p.h. and the wearable computer may recognizeambient noise as engine sounds, and thus recognize that the user isdriving a car. Thus, such abstract context questions can be moreintelligently answered using this contextual framework. Themes thatinclude sets of multiple related attributes further enhance thecontextual framework by permitting context clients or other code modulesto provide a quantification or qualification of a useful context of theuser that can not be directly measured from any attribute values in theset individually.

Simply providing a common exchange mechanism for useful information doesnot, in itself, promote interoperability. To fully realize separation oftasks and resulting interoperability, the context servers and contextclients will preferably speak a common language, and thus a commonnaming convention is used for attributes. Importantly, by standardizingattributes, context clients may share attributes provided by contextservers. Furthermore, themes provided by one or more context servers maybe processed by context clients independently of other themes processedby other context clients. Thus, a wearable computer, for example, maysimultaneously process multiple themes to answer numerous questionsregarding a user's context and provide a robust environment for theuser, as described below. In general, themes and their thematic sets ofattributes are created based on determining some or all of thefollowing: what types of context information are useful for a givenapplication, how context information accuracy is characterized byattributes, how to determine the validity or usefulness of an attributewith a given associated time value or timestamp, how to choose betweenmultiple providers of the same attribute, and what naming convention isto be applied to attributes.

The following discussion first introduces a suitable example wearablecomputer environment in which aspects of the invention can operate. Adetailed example of a cardiac condition theme and its possible use isthen provided. Thereafter, themes and their thematic attribute sets arediscussed more generally, and then examples of using themes to presentappropriate information to a user are illustrated.

Wearable Computer Example

Referring to FIG. 1, a context characterization system 100 is shownoperating in a general-purpose body-mounted wearable computer 120 wornby the user 110. FIG. 1 and the following discussion provide a brief,general description of a suitable computing environment in which theinvention can be implemented. Although not required, embodiments of theinvention will be described in the general context ofcomputer-executable instructions, such as routines executed by a generalpurpose computer. Those skilled in the relevant art will appreciate thataspects of the invention can be practiced with other computer systemconfigurations, including Internet appliances, hand-held devices,cellular or mobile phones, multi-processor systems, microprocessor-basedor programmable consumer electronics, network PCs, client-serverenvironments including thin clients, mini-computers, mainframe computersand the like. Aspects of the invention can be embodied in a specialpurpose computer or data processor that is specifically programmed,configured or constructed to perform one or more of thecomputer-executable instructions or modules explained in detail below.Indeed, the term “computer” as used herein refers to any data processingplatform or device.

Aspects of the invention can also be practiced in distributed computingenvironments, where tasks or modules are performed by remote processingdevices, which are linked through a communications network. In adistributed computing environment, program modules or sub-routines maybe located in both local and remote memory storage devices, such as withrespect to a wearable computer and a fixed-location computer. Aspects ofthe invention described below may be stored and distributed oncomputer-readable media, including magnetic and optically readable andremovable computer disks, as well as distributed electronically over theInternet or over other networks (including wireless networks). Thoseskilled in the relevant art will recognize that portions of theinvention may reside on a server computer or server platform, whilecorresponding portions reside on a client computer. For example, such aclient server architecture may be employed within a single wearablecomputer, among several wearable computers of several users, and betweena wearable computer and a fixed-location computer. Data structures andtransmission of data particular to aspects of the invention are alsoencompassed within the scope of the invention.

Many wearable computers (including the computer 120) travel with theuser, such as by being strapped or attached to a user's body or clothingor by being mounted in a holster. The wearable computer 120 has avariety of user-worn user input devices including a microphone 124, ahand-held flat panel display 130 (e.g., with a touch sensitive portionand character recognition capabilities), and various other user inputdevices 122. Similarly, the computer has a variety of user-worn outputdevices that include the hand-held flat panel display 130, an earpiecespeaker 132, an eyeglass-mounted display 134, a tactile output device136, and various other user output devices 138. In addition to thevarious user-worn user input devices, the computer can also receiveinformation from various user sensor input devices 126 and fromenvironment sensor input devices 128, including a video camera 121. Thecharacterization system 100, as well as various CCs and/or CSes (notshown), can receive and process the various input information receivedby the computer and can present information to the user on the variousaccessible output devices.

As shown in FIG. 1, the computer 120 is accessible to or communicateswith a computer 150 (e.g., by being in line-of-sight wireless proximityor by being reachable via a long-distance communication device such as acellular phone/modem) which also has a variety of input and outputdevices. In the illustrated embodiment, the computer 150 isnon-portable, although the body-mounted computer of the user cansimilarly communicate with a variety of other types of computers,including portable computers and body-mounted computers of other users.The devices from which the non-portable computer can directly receiveinformation include various user input devices 152 and various usersensor input devices 156. The non-portable computer can outputinformation directly to a display 160, a speaker 162, an olfactorydevice 164, and a printer 166. In the illustrated embodiment, thebody-mounted computer can communicate with the non-portable computer viaa wireless transmission medium. In this manner, the characterizationsystem 100 can receive information from the user input devices 152 andthe user sensor devices 156 after the information has been transmittedto the non-portable computer and then to the body-mounted computer.

Alternately, the body-mounted computer may be able to directlycommunicate with the user input devices 152 and the user sensor devices156, as well as with other various remote environment sensor inputdevices 158, without the intervention of the non-portable computer 150.Similarly, the body-mounted computer may be able to supply outputinformation to the display 160, the speaker 162, the olfactory device164, and the printer 166, either directly or via the non-portablecomputer, and directly to the telephone 168. As the user moves out ofrange of the remote input and output devices, the characterizationsystem 100 will be updated to reflect that the remote devices are notcurrently available.

The computers 120 and 150 can employ any known bus structures orarchitectures for coupling the various blocks of such systems, includingemploying a memory bus with memory controller, a peripheral bus, and alocal bus. Data storage devices (not shown) coupled to the computers 120and 150 may include any type of computer-readable media that can storedata accessible by a computer, such as magnetic hard and floppy diskdrives, optical disk drives, magnetic cassettes, flash memory cards,digital video disks (DVDs), Bernoulli cartridges, RAMs, ROMs, smartcards, etc. Indeed, any medium for storing or transmittingcomputer-readable instructions and data may be employed, including aconnection port to a network such as a local area network (LAN), widearea network (WAN) or the Internet.

The various input devices allow the characterization system 100 oranother system (not shown) executing on the computer 120 to monitor theuser and the environment and to maintain a model (not shown) of thecurrent conditions or context.

Such a context model can include a variety of attributes that representinformation about the user and the user's environment at varying levelsof abstraction. For example, information about the user at a low levelof abstraction can include raw physiological data (e.g., heart rate andEKG) and geographic information (e.g., location and speed), while higherlevels of abstraction may attempt to characterize or predict the user'sphysical activity (e.g., jogging or talking on a phone), emotional state(e.g., angry or puzzled), desired output behavior for different types ofinformation (e.g., to present private family information so that it isperceivable only to the user and the user's family members), andcognitive load (i.e., the amount of attention required for the user'scurrent activities). Background information which changes rarely or notat all can also be included, such as the user's age, gender and visualacuity. The model can similarly hold environment information at a lowlevel of abstraction, such as air temperature or raw data from a motionsensor, or at higher levels of abstraction, such as the number andidentities of nearby people, objects, and locations. The model of thecurrent context can additionally include information added explicitlyfrom other sources (e.g., application programs), as well asuser-specified or system-learned defaults and preference information. Anillustrative example of a context model containing user and environmentinformation is described in greater detail in U.S. patent applicationSer. No. 09/216,193, filed Dec. 18, 1998 and entitled “Method and Systemfor Controlling Presentation of Information to a User Based on theUser's Condition.”

Those skilled in the art will appreciate that computer systems 120 and150, as well as their various input and output devices, are merelyillustrative and are not intended to limit the scope of the invention.The computer systems may contain additional components or may lack someof the illustrated components. For example, it is possible that thecharacterization system 100 can be implemented on the non-portablecomputer 150, with the body-mounted computer 120 replaced by a thinclient such as a transmitter/receiver for relaying information betweenthe body-mounted input and output devices and the non-portable computer.Alternately, the user may not wear any devices or computers.

In addition, the body-mounted computer 120 may be connected to one ormore networks of other devices through wired or wireless communicationmeans. In general, as used herein, communications or data exchangebetween the devices or components described herein may be performedusing any wired or wireless methods, (e.g., wireless RF, wirelesssatellite connections, a cellular phone or modem, infrared, physicalcable or wiring of any variety, a docking station, physical contextbetween two WPC users, etc.), either with or without support from othercomputers such as the computer 150. For example, when loaded with a“home” attribute set and corresponding CSes and CCs, the body-mountedcomputer of a user can make use of output devices in a smart room (e.g.,a television and stereo when the user is at home), with the body-mountedcomputer transmitting information to those devices via a wireless mediumor by way of a cabled or docking mechanism if available. Alternately,kiosks or other information devices can be installed at variouslocations (e.g., in airports, stores, a work place, or at tourist spots)to transmit relevant information to body-mounted computers within therange of the information device.

In general, as the body-mounted computer receives various inputinformation, the information is forwarded to the characterization system100. The characterization system 100 monitors the user and the user'senvironment in order to create a current user context model. Inparticular, the characterization system 100 receives a variety of typesof information, and can use this information to determine the user'scurrent context in a variety of ways. These types of information includeexplicit user input to the computer (via input devices 122, etc.),sensed user information (via user sensors 126, etc.), and sensedenvironment information (via environment sensors 128, etc.). Thecharacterization system 100 can also receive date and time informationfrom a CPU or from some other source, and can retrieve storedinformation (e.g., user preferences, definitions of various user-definedgroups, or a default model of the user context) from a storage device(not shown in FIG. 1). It is also possible for one or more applicationprograms to optionally supply application-specific information to thecharacterization system 100. This information can include any type ofuser context information to which the application program has access,such as user location or physiological state. In addition, theapplication programs can create new user context attributes, includingthose to be used only by that application program. All of the aboveitems of information may be used to generate attribute values by contextservers.

The various input information can provide context information in avariety of ways. For example, user input information alone can providesignificant information about the user's context. If the user iscurrently supplying input to the computer via a full-sized keyboard, forinstance, it is likely that the user is engaged in little other physicalactivity (e.g., walking), that the user is devoting a significant amountof attention to the computer system, and that the user would seeinformation flashed on the display. If the user is instead generatinguser input audibly (e.g., through the head-mounted microphone 124), thatfact may provide less user context information since the user can supplysuch audio information while engaged in a variety of types of physicalactivity. Those skilled in the art will appreciate that there are a widevariety of input devices with which a user can supply information to thecomputer system, including voice recognition devices, traditional qwertykeyboards, chording keyboards, half qwerty keyboards, dual forearmkeyboards, chest mounted keyboards, handwriting recognition and digitalink devices, mice, track pad, digital stylus, finger or glove devices tocapture user movement, pupil tracking devices, gyropoints, trackballs,joysticks, game pads, scanners (including optical character recognition(OCR) scanners and barcode scanners and other automated data collectionreaders), radio-frequency identification (RFID) readers, voice griddevices, video cameras (still and motion), etc.

In addition to the information received via user input, thecharacterization system 100 also uses sensed information about the user(from, e.g., the user sensors 126). For example, a variety of sensorscan provide information about the current physiological state of theuser, geographical and spatial information (e.g., location andaltitude), and current user activities. Some devices, such as themicrophone 124, can provide multiple types of information. For example,if the microphone 124 is available, the microphone can provide sensedinformation related to the user (e.g., detecting that the user istalking, snoring, or typing) when not actively being used for userinput. Other user-worn body sensors can provide a variety of types ofinformation, including that from thermometers, sphygmometers, heart ratesensors, shiver response sensors, skin galvanometry sensors, eyelidblink sensors, pupil dilation detection sensors, EEG and EKG sensors,sensors to detect brow furrowing, blood sugar monitors, etc. Inaddition, sensors elsewhere in the near environment can provideinformation about the user, such as motion detector sensors (e.g.,whether the user is present and is moving), badge readers, video cameras(including low light, infra-red, and x-ray), remote microphones, etc.These sensors can be both passive (i.e., detecting information generatedexternal to the sensor, such as a heart beat) or active (i.e.,generating a signal to obtain information, such as sonar or x-rays). Allof the above items of information may be used to generate attributevalues by context servers.

Stored background information about the user can also be supplied to thecharacterization system 100. Such information typically includesinformation about the user that changes at most infrequently, althoughit is possible to frequently update the stored background information toreflect changing conditions. For example, background information aboutthe user can include demographic information (e.g., race, gender, age,religion, birthday, etc.) if it can affect when and how contextinformation is created and used. User preferences, either explicitlysupplied or learned by the system, can also be stored as backgroundinformation. Information about the user's physical or mental conditionwhich affects the type of information which the user can perceive andremember, such as blindness, deafness, paralysis, or mentalincapacitation, is also important background information that allowssystems with access to this information to adapt to the user'scapabilities and to create and use appropriate context information.

In addition to information related directly to the user, thecharacterization system 100 also receives and uses information relatedto the environment surrounding the user. For example, devices such asmicrophones or motion sensors may be able to detect whether there areother people near the user and whether the user is interacting withthose people. Sensors can also detect environmental conditions which mayaffect the user, such as air thermometers, Geiger counters, chemicalsensors (e.g., carbon monoxide sensors), etc. Sensors, eitherbody-mounted or remote, can also provide information related to a widevariety of user and environment factors including location, orientation,speed, direction, distance, and proximity to other locations (e.g., GPSand differential GPS devices, orientation tracking devices, gyroscopes,altimeters, accelerometers, anemometers, pedometers, compasses, laser oroptical range finders, depth gauges, sonar, etc.). Identity andinformational sensors (e.g., bar code readers, biometric scanners, laserscanners, OCR, badge readers, etc.) and remote sensors (e.g., home orcar alarm systems, remote camera, national weather service web page, ababy monitor, traffic sensors, etc.) can also provide relevantenvironment information. All of the above items of information may beused to generate attribute values by context servers. Informationregarding storing and retrieving environmental information may be foundin U.S. patent application Ser. No. 09/464,659, filed Dec. 15, 1999 andentitled “Storing and Recalling Information to Augment Human Memories.”

In addition to receiving information directly from low-level sensors,the characterization system 100 can also receive information frommodules which aggregate low-level information or attributes intohigher-level attributes (e.g., face recognizer modules, gesturerecognition modules, affective/emotion recognizer modules, etc.). Asexplained below, a combined context server and context client module mayreceive attribute values from low-level sensors and produce higher-levelattribute values which are then fed back into the characterizationmodule (CM) for use by other context clients.

A user can also explicitly supply information about their currentcontext (e.g., “I have a high cognitive load and do not want to bedisturbed” or “I am distracted and will need greater assistance thannormal in recalling current state information”). The characterizationsystem 100 can also receive current date and time information in orderto both track changes over time and to utilize information such as theuser's stored schedule. Previously-created models of the user's contextcan also be retrieved and used as a default or to detect changingconditions. Information from the computer indicating the types of outputcurrently being presented to the user can also provide information aboutthe user's current context, such as current activities and cognitiveload.

After the characterization system 100 receives one or more of thesetypes of information, it processes the information and creates a currentmodel of the user context based on multiple attributes (with currentvalues for some or all of the variables). Once the model of the usercontext has been created and then later updated, older versions of themodel will be made available to the characterization system 100 toassist in the characterization of the user, such as with changes overtime.

The model of the current user context can represent a variety of typesof information. The characterization system 100 may merely store theattribute values it receives (even when they are at a low-level ofabstraction) and then allow other modules to directly use the storedattributes when making decisions related to the current state. Thecharacterization system 100 may also use received low-level attributesto itself generate attributes for higher-level representations of theuser's observable activities (e.g., walking, watching a movie in a movietheater, talking to co-workers at the office, etc.).

In yet another embodiment, the characterization system 100 furthercharacterizes the user's context with respect to attributes that are notdirectly observable. Such attributes include the current cognitive loadof the user (indicating amount of attention required for the user'scurrent activities and thus the ability of the user to devote attentionto the computer), the current degree of interruptibility for the user(indicating ability to safely interrupt the user), the current degree ofintrusiveness of output on the environment (indicating impact of outputon the surrounding environment), the user's desired scope of audiencefor information being output (indicating how many people should be ableto perceive the information), the user's desired level of privacy forinformation being output (indicating the group of people who are allowedto perceive the information), and the user's desired level of solitude(indicating the user's current desire to avoid intrusions).

User attributes can also represent abstract principles about the userand the surrounding environment, such as the user's relationship toother objects, people, or locations (e.g., being at their desk, being intheir office, being near a drug store, talking to a particular person,etc.). In some embodiments, modules in the characterization system 100can create and supply information about user context attributes to othermodules, and those other modules can add the information to their modelof their user context if appropriate (e.g., ambient air temperature, oran emotional state of a module's user that is sensed by another module),as noted below.

EXAMPLE Cardiac Condition Context Client and Associated Context Serversand Attributes

Referring to FIG. 2, a more detailed block diagram of an embodiment ofthe computer 120 and of the characterization system 100 is shown. Inparticular, FIG. 2 shows an example of a set of input devices, contextservers, and associated attributes that employ a cardiac conditionthematic attribute set to characterize the user's cardiac condition. Thetheme of the attribute set employed under the example of FIG. 2 is thusthe cardiac condition of the user. A cardiac condition client thenemploys the cardiac condition attribute set to model a current cardiaccondition of the user, where such a condition could not be directlymeasured from any of the attributes individually.

In the illustrated embodiment, the computer 120 receives input from notonly the microphone 124 and video camera 121, but also from a bodytemperature sensor 202, EKG/EEG sensor 204, heart rate monitor 206,blood pressure monitor 208, blood oxygen sensor 210, and ambienttemperature sensor 212. Of course, other input devices related todetermining a user's cardiac condition may also be used, such as a skingalvanometer. The input devices each provide one or more data signals tocorresponding context servers (CSes) 214-224, which receive such signalsand convert them into attributes for use by context consumers or clients(CCs). For example, an audio input CS 214 receives an audio data signalfrom the microphone 124, and then parses, amplifies/attenuates, filters,packages or otherwise processes such a signal to produce a soundattribute.

If the microphone 124 is for receiving speech commands, then the audioinput CS 214 can continually monitor audio data signals produced by themicrophone 124 until a data signal is received that exceeds apredetermined threshold, which may indicate that the user is speaking.Such a signal is then parsed into appropriate packets such as phonemes,words or phrases to produce audio command attributes. Conversely, if themicrophone 124 is instead positioned for receiving ambient sounds, thenthe audio input CS 214 may be modeled to simply take samples at regularintervals of the ambient audio data and convert such signals into anaudio environment attribute. Of course, two or more microphones may beprovided for receiving both spoken commands and ambient sounds, or onemicrophone may pick up both types of audio data. As explained below,such an audio environment attribute may be used by a CC to determineinformation about the user's context, such as whether the user isexercising at a health club or sleeping.

Other CSes shown in FIG. 2 provide similar processing of data signalsprovided by input devices. For example, a video input CS 216 mayperiodically sample and process frames of video data provided by thevideo camera 121. A body temperature CS 218 receives input signals fromboth the body temperature sensor 202 and the ambient temperature sensor212, and then compares the signals to provide a close estimate of auser's core body temperature and to produce a corresponding bodytemperature attribute value.

Two heart rate CSes 220 are shown in FIG. 2 such that one heart rate CSreceives an input signal from the EKG sensor 204 while the otherreceives an input data signal from the heart rate monitor 206. The firstheart rate CS 220 may provide a more accurate heart rate attribute basedon the EKG input data signal, while the second heart rate CS may providean alternate heart rate attribute, with either or both of the heart rateattributes being used by a CC.

In a manner similar to that described above, a blood pressure CS 222receives an input data signal from the blood pressure monitor to createand provide a blood pressure attribute value, while a blood oxygen CS224 receives a blood oxygen input signal and provides a blood oxygenattribute value in response. Other CSes 226 are also shown which canreceive input data signals from the various user sensor devices 126,environmental sensor devices 128, and user input devices 122, and canprovide corresponding attribute values.

The various CSes 214 through 226 provide attribute values to acharacterization module (CM) 230 that acts as an exchange mechanism forattribute values from CSes to CCs. The CM 230, CSes 214 through 226, andCCs reside in a memory 232 that forms part of the computer 120. Alsorunning or resident in the memory 232 are one or more applicationprograms 236 (e.g., word processing and spreadsheet applications), and abrowser program 238 such as for accessing the World Wide Web portion ofthe Internet. The memory 232 may be random access memory (RAM),electronically erasable read only memory (EEPROM), data storage devicessuch as floppy and fixed magnetic disks, optical disks, or any otherdata storage devices, and any combination of such memory.

The computer 120 also includes one or more central processing units(CPUs) 234. The term “CPU,” as generally used herein, refers to anylogic processing unit, such as one or more microprocessors, digitalsignal processors (DSPs), application-specific integrated circuits(ASIC), programmed logic arrays and the like. While the CPU is shown asa block separate from other blocks, such as the memory, some or all ofthese blocks may be monolithically integrated onto a single chip.

A general discussion of the interrelationship between context servers,context clients and the characterization module 230 will now bedescribed. The CM 230 provides a mechanism for attribute exchangebetween code modules such as applications, dynamic link libraries(DLL's) and drivers. As shown in FIG. 3, a basic architecture 300includes the CM 232 and code modules A through D. As noted above, amodule that provides attributes to the CM is referred to as a server ofattributes, while a consumer of attributes is referred to as a client.In FIG. 3, code module A is a server (CS) while code module B is aclient (CC). Clients and servers differ only in the direction in whichattributes flow. Modules may act as both clients and servers if theyboth provide and consume attributes (e.g., code modules C and D).

Context clients may look at or employ as many attribute values as theywish, regardless of the source or server of such attributes. Under analternative embodiment, the CM may include mechanisms that restrictaccess to certain attributes, both for clients and servers, such as forsecurity reasons or to prevent circular references.

Context clients and servers inform the CM of their intent to communicateattributes through a registration process. This registration processprovides the CM with a way to monitor attribute dependencies, to notifyparticipating modules of specific events, and to establish precedents.Further details on interaction between context clients, servers and theCM may be found in U.S. patent application Ser. No. 09/541,328, entitled“Interface for Exchanging Context Data” and filed Apr. 2, 2000. Oncecontext clients have registered their intent to examine or receiveattributes, such clients may create event handlers so that they receivenotification from the CM when attributes change in specific ways, suchas exceeding a predetermined threshold. Further details on event andcondition-driven attribute creation/supply may be found in U.S. patentapplication Ser. No. 09/724,892, filed Nov. 28, 2000 and entitled“Automated Response To Computer User's Context.” Further details on theCM supplying attributes for user context data, and employing such datamay be found in U.S. patent application Ser. Nos. 09/724,902,09/724,893, 09/724,777, 09/724,894, 09/724,949, and 09/724,799, andentitled “Dynamically Exchanging Computer User's Context,” “SupplyingEnhanced Computer User's Context Data,” “Requesting Computer User'sContext Data,” “Managing Interactions Between Computer Users' ContextModels,” “Supplying Notifications Related To Supply And Consumption OfUser Context Data,” and “Generating And Supplying User Context Data”respectively, and all filed Nov. 28, 2000.

A context client may request an attribute that more than one contextserver provides. Each contributing context server is said to provide aninstance of an attribute. The context client may either specify thepreferred server/instance, or it may ask the CM to mediate the multipleattribute instances. The mediation process involves implementing one ofseveral defined strategies for evaluating the instances and sending aresulting aggregate or representative instance of an attribute to therequesting client. These strategies are referred to as attributemediators. As shown in FIG. 2, the CM 230 includes a mediate process 240which receives heart rate attributes from the two heart rate CSes 220.The mediate process 240 may select between such two heart rateattributes, such as to select the most accurate (e.g., the one producedfrom the EKG sensor 204) or the newest, or may average between suchattributes. Further details on such a mediation process may be found inU.S. patent application Ser. No. 09/724,932, entitled “MediatingConflicts In Computer User's Context Data” and filed Nov. 28, 2000.

The CM makes no distinction between directly measured attributes andderived attributes. For instance, while it is simple to both quantifyand measure a user's pulse or heart rate, the user's level of physicalactivity and cardiac condition may require inference from multiplemeasurable quantities such as pulse, velocity (as measured from GPSdata), skin galvanometry (as measured from a skin galvanometer) and thelike, as explained in greater detail below. The context clients receivesuch attributes from the CM to characterize a current state of the user,and in some cases, to provide output (e.g., by displaying information)to the user.

Referring to FIG. 4, a simple linear routine 400 summarizes how one ormore context servers obtain input signals and provide such signals tocontext clients via the CM in order to provide an output to the user,particularly with respect to a thematic data set to represent orcharacterize the user's context (e.g., cardiac condition). Under step402, one or more context servers and context clients are initialized fora thematic data set for the user. Such initialization may includeregistration of the CS and CC modules with the CM.

In step 404, the CS obtains data signals from input devices. In step406, the CS processes the input data signals to produce attributesrepresenting the user's context, including adding required data fieldssuch as attribute name, timestamp, units, uncertainty and the like.

In step 408, the CS provides such processed attributes to the CM forstorage. Under step 410, the CC requests from the CM to receive thestored attributes. Under step 412, the CC processes the receivedattributes to determine and model the user context and to determine anyappropriate output. Under step 414, the CC, in conjunction with anyappropriate drivers, formatting modules and output devices, provides anoutput signal to the user or other entity. Alternatively, the CC mayprovide such context information or processed attributes to anothermodule or process.

Applying the routine 400 of FIG. 4 to the illustrated embodiment of FIG.2, the input devices provide data signals to the CSes 214 through 226,which in turn produce attributes for storage by the CM 230 (under steps402 through 408). A cardiac condition CC 242 requests and receives fromthe CM the set of various cardiac condition attributes, including anambient audio attribute, current video attribute, body temperatureattribute, mediated heart rate attribute, blood pressure attribute, andblood oxygen level attribute, as well as other attributes not shownunder step 410. Other relevant attributes may include the user's moodand the user's speech patterns to determine whether an urgent cardiaccondition exists or not. (Of course, other CCs 244 may also request andreceive these and other attributes.) Indeed, the heart rate, bloodpressure, blood oxygen level and body temperature attributes, as well asother attributes, represent a cardiac condition data set or “thematicattribute set”. The cardiac condition CC 242 processes the cardiaccondition attribute set to model a characteristic of the current stateof the user's cardiac condition. For reasons of economy and clarity, thecardiac condition CC 242 is described below as a single context clientproviding numerous functions; those skilled in the relevant art willrecognize that the cardiac condition CC may well employ numerous modulesor subroutines, or other CCs, to provide the functionality describedbelow.

The cardiac condition CC analyzes all received attributes in the cardiaccondition attribute set to determine a best estimate of the user'scurrent cardiac condition or state, and to determine a best output orresponse to that state. For example, the heart rate, blood pressure, andblood oxygen level attributes may show elevated heart levels, high bloodpressure and low oxygen levels respectively, which may indicate that theuser is undergoing cardiac stress. However, the cardiac condition CC mayalso analyze the ambient audio attribute, visual elements within a frameof video data or other video input attributes, and the body temperatureattribute. The ambient audio attribute may reflect heavy breathing ofthe user, the body temperature attribute may reflect an elevated surfacebody temperature, and the video data may include visual clues reflectingathletic equipment. Thus, the cardiac condition CC can determine thatthe user is exercising at the health club. Further, the cardiaccondition CC may retrieve the user's electronic calendar or schedule(not shown) stored in the memory 232, which may indicate that, basedupon the current time, the user is scheduled to be exercising. Thus, thecardiac condition CC decides to provide an output to the user indicatingthe user's current level of physical activity. The cardiac condition CCoutputs a signal to a formatter and visual display selector 248 thatprovides an appropriate signal to the head-mounted display 134 tovisually indicate the user's activity level (e.g., a “tachometer”-typedisplay having a needle representing a numerical value on the tachometercorresponding to the user's activity level).

If, however, the heart rate, blood pressure and blood oxygen levelattributes indicate a possibly dangerous cardiac condition, and otherattributes in the cardiac condition attribute set don't attenuate thisindication but instead further support a possible serious condition,then the cardiac CC provides warnings or possible emergency output tothe user. For example, if the cardiac condition CC 242 determines thatthe user is about to have a heart attack, the CC provides both a visualoutput signal to the head-mounted display 134 to provide a warning tothe user and an audible warning signal to the earpiece speaker 132. Ifin fact the user is having a heart attack, then the cardiac condition CC242 may provide an output signal to a defibrillator 250. The measuredattributes or cardiac condition data set allow the cardiac condition CCto determine an appropriate charge for the defibrillator 250 to apply tothe user. Indeed, the cardiac condition data set is generally morerobust than individual attributes, and thus the cardiac condition CC canprovide a much more accurate charge determination for the defibrillator250 than prior art defibrillators.

The cardiac condition CC 242 may also call for help if the user issuffering from a serious cardiac condition. Under this example, thecardiac condition CC 242 must first determine a current state of theuser's logical data and telecommunications environment. For example, oneof the various sensor devices 128 may include a cell phone signaltransceiver to determine whether a cell phone site is within range ofthe user, and possibly to register the user with the cell site. One ofthe other CSes 226 then creates a proximate telecommunications accessattribute. The cardiac condition CC 242 receives such an attribute todetermine if the cardiac condition CC may provide an output signal to atelecommunications transceiver 252, and if so dials an emergency numberor 911 and automatically provides audio notification with a currentlocation of the user (e.g., using GPS location). If the user is notwithin a cell site (as determined by the proximate telecommunicationsattribute), the cardiac condition CC may then cause thetelecommunications transceiver 252 to employ alternativetelecommunications devices, such as paging a doctor or a hospital via apaging network. Other attributes reflecting a state of the user'slogical data and telecommunications environment include the above-notedelectronic calendar for the user. The cardiac condition CC 242 may alsoretrieve important phone numbers stored in a contacts database for theuser so that the cardiac condition CC may place a call or page to aparticular doctor or other health care provider.

Based on attributes reflecting a state of the user's physicalsurroundings, the cardiac condition CC may provide additional output.For example, the cardiac condition CC 242 may analyze the ambient audioinput attribute to determine whether voices or sounds of nearby peopleare present. If so, the cardiac condition CC may provide an outputsignal to a speaker 254 that will broadcast prerecorded messages such as“I'm having a heart attack! Please call for help!” to thereby allow thewearable computer to call for help from nearby people when the user maybe incapacitated.

To summarize, the cardiac condition CC 242 employs and analyzes athematic set of attributes reflecting three distinct states of the user:(1) the state of the user (biometric, mood, etc.), (2) the state of theuser's physical surroundings, and (3) the state of the user's logical ortelecommunications environment, to thereby produce an appropriateresponse. In other words, the cardiac condition thematic attribute setincludes not only attributes reflecting the state of the user (e.g.,heart rate, blood pressure, body temperature and blood oxygen levelattributes), but also attributes regarding the state of the user'sphysical surroundings (e.g., the ambient audio input attribute) and theuser's logical or telecommunications environment (e.g., whether thecomputer 120 may initiate a cellular telephone call using a proximatetelecommunications access attribute). Of course, the cardiac conditionthematic set for use with the cardiac condition CC may include otherattributes not discussed.

More generally, a user's logical or telecommunications environmentrepresents data connectivity or a data environment indicating how theuser's computer may have access to or connect with other computers ordata sources. The user's logical environment also provides informationregarding existing data stored elsewhere, such as emails, voice mails,phone calls, facsimiles, commonly used databases, and the like. Forexample, as noted above, attributes reflecting a state of the user'slogical and telecommunications environment include an electroniccalendar and contacts database, such as that provided by MicrosoftCorporation under the trademark Outlook. Other attributes reflecting astate of the user's logical or telecommunications environment includefavorite Internet links or web pages, phone numbers or othertelecommunications access addresses (“resource locators”) identifying atelecommunications node, including web pages, databases, documents,facilities, resources, and other data and processes accessible to theuser via the user's computer 120.

As explained below, numerous thematic data or attribute setscharacterizing the user's context are possible, all of which aregenerally based on these three classes of user context (i.e., the user'sstate, physical surroundings, and logical data and telecommunicationsenvironment). Attributes based on these three context classes providerich context information for context clients to model characteristics ofthe current state of the user and to produce a response that accuratelyreflects the user's context in specialized scenarios or for specializedtasks. Prior art devices and processes that receive only a single signalor a small set of input data signals can produce only limited responses,and thus fail to fully characterize the user's context.

Thematic Attribute Sets, and Generation and Use of Such Sets

Initially presented below is a discussion of how attribute values arerequested by the CM, created by context servers and provided to thecontext clients, as well as an overview of a structure for attributes.Two examples are then provided. Thereafter, properties of attributes,their uses and thematic sets of such attributes are discussed.

Under one embodiment, the characterization module requests attributesfrom each context server by sending a message to an appropriate requesthandler running within each context server. The request includes thename of the attribute being requested and a timeout period. The contextserver is expected to provide an attribute within the timeout period,where such attribute includes the following: value or quantity of theattribute; an uncertainty value that represents a range of values aroundthe attribute value that the attribute is likely to have, with the typeof the uncertainty value depending upon the type of information embodiedin the attribute value; a timestamp that represents the effective age ofthe attribute value; and an indication of the measurement units in whichthe value and uncertainty are expressed. The timestamp reflects a pointin time when the attribute value is most accurate, and it is theresponsibility of the context server to determine how to timestamp itsattributes most effectively. Attributes may also include a formatversion, as well as various flags. The timestamp may be determined fromthe clock residing on the computer 120 to facilitate consistency andeasy comparison between context clients.

CSes may create attribute values at any time. An attribute valuecreation request from the CM includes the following: a CS name,attribute name, data type, format version, request handler and startupbehavior. The CS name is a name for the requesting CS that is uniqueamong other CSes. The name should be the same for all attributes thatthe CS creates. The attribute name corresponds to a predetermined namefor the attribute. The data type represents the manner in which theattribute's value and uncertainty are expressed, such as floating pointor text string data types. The format version represents the number ofthe format version in which the value is expressed. The request handleris a reference to a CS function that processes attribute requests andother messages from the CM. A single request handler may also be usedfor multiple attributes. The startup behavior specifies whether or notthe CS should be loaded automatically at startup. This parameter isoptional, and if included it supersedes any previous setting for therequesting CS. Overall, sets or groups of attributes generated by one ormore CSes related to a common theme are provided and stored in the CMfor use by one or more CCs.

A CC may request from the CM a value of an attribute within a thematicset. Such a request includes the following: a name, source, attributemediator, age and timeout. The name represents a string that identifiesthe attribute. The source is the name of the CS that is to provide theattribute value. If no source is specified, then the CM assumes theattribute value may come from any source, and if necessary it uses anattribute mediator in the CM to select one. As noted above, theattribute mediator is the name of the selection method that the CC wouldlike the CM to use when selecting one of multiple attribute instances.If no attribute mediator is provided, a default method is used. The ageis an expression of the maximum age of the attribute value. If therequest is made for a specific attribute instance through specificationof a source, then only that instance is checked, and if that attributevalue is too old it is freshened by a request from the CM to the CS fora new attribute instance value. If multiple instances are present and asource is not specified, the CM applies an attribute mediator to thoseattributes whose values satisfy the age requirement. If no valuessatisfy the age requirement, all instances of the attribute arefreshened prior to the application of the attribute mediator. Thetimeout is a period within which the CM shall return the attribute.Attributes may also be requested periodically; further details may befound in U.S. patent application Ser. No. 09/541,326, entitled “Loggingand Analyzing Computer User's Context Data” and filed Apr. 2, 2000.

In response to the above attribute requests, the CM provides the CC withthe following for each attribute: the attribute's name, value,uncertainty, timestamp, units, source, and attribute mediator. It mayalso include format version and flags. The name, value, uncertainty,timestamp, units and attribute mediator are described above. The sourceinformation refers to the object that created and owns the instance ofthe attribute, typically a CS. If the attribute is derived from anevent, the source is the name of that event. Other attributes have theCM as their source. Further details on functionality for CSes, CCs andthe CM, including attribute creation and use, are discussed in theabove-referenced U.S. patent applications.

Additional functionality for CCs and CSes is also possible. For example,CCs may obtain listings of attributes, and register with the CM toreceive notification when attributes that they request become available.CSes may also be provided with notification when clients register forattributes whose values they provide, thereby allowing such CSes to loadinto memory and execute only when necessary. The CM may also in somesituations convert attribute values from one measurement unit toanother. CCs may also be notified that data cannot be newer than acertain preset period of time (attribute instance minimum delaycommunication). In addition, certain CSes may have exclusivity inproviding values for certain attributes, or may push attribute values tothe CM (e.g., for frequently updated attributes). CSes may also have theability to provide constant attributes whose values do not change, andthus that do not require repeated requests from the CM for the currentattribute value. Further, CSes may in some situations be required as asecurity precaution to be certified, and in other situations attributenames may be reserved so that CSes cannot use those names.

In general, CSes may create low-level, intermediate-level andhigher-level attributes within the framework noted above. For example,referring to FIG. 5A, a latitude, longitude and altitude context servermodule or routine 500 is shown. The CS module 500 provides a thematicset of three attributes related to location, those being latitude,longitude and altitude. Beginning in step 502, the module launches.Because the CS module functions independently of any particular contextclient, this context server module launches at system startup.Additionally, under step 502, the module 500 initiates alatitude/longitude/altitude attribute request handler and registers themodule and the attributes with the CM. In step 504, the CS module 500periodically receives a GPS data stream from a GPS receiver thatrepresents a most recent location of the user (assuming that the GPSreceiver forms part of the wearable computer). In step 506, the moduleparses and stores the received GPS data.

In step 508, the CS module 500 determines whether it has received arequest from the CM for a latitude, longitude or altitude attribute, orany combination of these attributes. If not, then the module loops backto step 504. If yes, then the module provides the requested attributevalues to the CM together with timestamp, uncertainty, units and otherfields or portions of the attribute. Referring to FIG. 5B, an example ofthe latitude attribute is shown as a data structure 520. The latitudeattribute includes a latitude name 521 (e.g., “Latitude”), a latitudevalue 522, an uncertainty value 524, a timestamp 526, units 528 and aformat version, flags or other data 530, all of which are discussedabove. Data structures for the longitude and altitude attributes in thislocation attribute set are substantially similar.

Context servers may also produce intermediate-level attributes which arecalculated from lower-level attributes for a thematic attribute set. Forexample, a speed attribute, representing the speed of the user, can becalculated directly by a sensor such as a pedometer (thus being alower-level attribute), or can be calculated indirectly via informationover time from the latitude, longitude and altitude attributes (via theGPS sensor). Other calculated attributes include an indication that theuser is located in his or her office, is near his or her desk, and thatthere are no other people physically nearby. Thus, a thematic attributeset reflecting whether other people are present may include a motiondetection attribute based on a signal provided by a motion sensor devicelocated on the desk of the user that determines whether otherindividuals are in the user's office, and an ambient sound attribute todetermine whether other voices may be heard within the user's office.

Higher-level condition variables or attributes can also be calculatedand be included within thematic attribute sets, such as the user'scurrent physical activities, the current user cognitive load, thedesired level of privacy, and the desired scope of audience. Informationfrom a microphone or directly from a cellular phone can indicate thatthe user is currently talking on their cellular phone, and the speed andmotion sensor data can indicate that the user is walking. Since the userremains near his desk even though he is walking, the system can deducethat the user is pacing about his office or is walking on a treadmill.Such a user activity attribute demonstrates that variables can havemultiple values, and that information such as a degree of belief orcertainty in the value for a variable can be added and used by thesystem.

Referring to FIG. 6, an example of a context client (CC) is shown forusing the low-level location attribute set provided by the module 500 ofFIG. 5A. Beginning in step 602, a message handler in a location module600 registers with the CM as having an interest in values of thelatitude, longitude and altitude attributes. In step 604, the CC module600 determines whether to operate under normal operation. If so, thenthe module accepts the default attribute mediator in the CM forproviding the latitude, longitude and altitude attributes. If not, suchas when the user desires a more accurate latitude attribute value, themodule 600 in step 608 requests the CM to reevaluate all latitudeattribute values from all CSes providing such data. Other types of CSmodules able to produce latitude, longitude and altitude coordinatesinclude location finding methods using cell sites, card readers oraccess entry points (particularly within a building or buildingcomplex), LORAN devices and the like. In step 610, the module 600requests that the CM provide the latitude attribute whose value has thelowest uncertainty by specifying an attribute mediator that selects theattribute having the lowest uncertainty value. In step 612, the module600 receives the lowest uncertainty latitude values and determines whichCS provided the value.

In step 614, the module 600 requests values for the latitude, longitudeand altitude attributes from the CM under default mode and determineswhether it has received such attribute values. Alternatively, under ahigh accuracy mode, the module could request latitude, longitude andaltitude attributes from the specific CS that provided the highestaccuracy latitude attribute above (assuming that the specific CSprovides values for all three of the attributes), and receive suchattribute values in this step. The module 600 may also provide the setof location attributes it receives under step 614 to another module orprocess if appropriate, such as for indicating a current position of theuser on a map image displayed on the head-mounted display 134 or flatpanel display 130.

In step 616, the module 600 determines whether a notification feature ofthe module is active. Such a feature could be used to indicate whetherthe user has entered or exited a predefined area or boundary line. Ifsuch a feature is active, then in step 618 the module 600 receives userinput defining boundary conditions, such as latitude and longitudecoordinates defining a rectilinear area or simply coordinatesidentifying a single street or boundary line. For each such boundary,the module 600 creates corresponding conditions in the CM on thelatitude and longitude attributes. While the module could insteadperiodically request and compare received attribute values to theboundaries defined under step 618, the use of CM conditions allows themodule to offload this processing to the CM who can periodically check(e.g., every two seconds) to see if conditions have changed and theboundary has been crossed. Therefore, the module 600 in step 622 employsa message handler that simply determines whether a response is receivedfrom the CM based on the previously established condition. If so, thenin step 624 the module notifies the user that the boundary was crossed,such as by providing an audio output to the earpiece speaker 132 or anunobtrusive visual display on the headmounted displayed 134. If themodule 600 employed a map attribute, then the location thematicattribute set could be expanded beyond, latitude, longitude and altitudeattributes, to include map attributes (as well as other attributes thatthe module may employ).

The CS module 500 and CC module 600 are examples of simple contextsources and clients; those skilled in the relevant art will readilyrecognize that more detailed modules with enhanced functionality can becreated based on the teachings provided by such examples. For example,the CC module 600 could further include route-finding capabilities basedon a specified destination. In particular, the CC 600 could present alist of predefined named places to the user via the head-mounted display134, and allow the user to select a desired destination. The CC 600could then identify the user's current location via attributes providedby the CS 500. The current location and destination point could then besent via wireless Internet connectivity to a website that providesroute-finding services. A map display CS could use the generated routeto provide a map attribute having appropriate map image value to the CC,which could then display the map via the head-mounted display 134.Alternatively, the route-finding service may provide step-by-step textdata listing directions to the destination, which when received by theCC could similarly be displayed to the user. The CS and CC could also beconfigured to update directions as the user moves from the currentlocation to the destination, or to provide a new set of directions ifthe user desires a new course (e.g., such as due to traffic, timeconstraints to reach an intermediate destination and the like). Underthis example, the CC module 600 employs not only the location attributeset (including latitude, longitude and altitude attributes), but also aset of place name attributes, map display or route-finding serviceattributes (e.g., Internet links or resource locators), and the like—allof which form a thematic attribute set where the theme represents “routefinding.”

The CC module could also continually monitor the user's location anddetermine what named places are near. If the user has tasks associatedwith locations, the CC can notify the user when the user is approachinga place with an associated task, such as via the head-mounted display134 or earpiece speaker 132. Tasks can also has associated priorities,and the notification could be conditional based on the task's prioritylevel. In this manner, lower-priority tasks could be ignored or provideonly minimal output on the head-mounted display 134, whereas higherpriority tasks could provide greater visual display on the head-mounteddisplay and/or an audible notification on the earpiece speaker 132. Asan example, the user creates a new task entitled “Grocery List,”together with information about the task such as all grocery itemsneeded. The user than specifies that the task belongs to a givencategory, such as “Groceries.” The category “Groceries” then has a placepredefined, such as a favorite grocery store or a grocery storeidentified along a regular route (e.g., the route traveled from work tohome). The CC then monitors the user's current location via latitude andlongitude coordinates provided by the CM, and calculates correspondingdistance to the grocery story. When the distance falls below a thresholdvalue, the CC identifies tasks for the given category, such as thegrocery list, and provides a notification to the user. Thus, under thisexample, the CC module employs a thematic data set including not onlythe location attribute set noted above, but also task attributes(representing links or resource locators for the user's stored list oftasks, including the grocery list)—which together represent a thematicdata set where the theme is “tasks.”

The CC module 600 or other context client/server modules may also bedistributed to users on computer-readable media, such as optically ormagnetically readable disks. The context client modules may be providedtogether with one or more context server modules, such as the CS module500, or each module may be supplied separately. CC and CS modules couldalso be provided electronically (e.g., downloaded via a wired orwireless connection from the Internet) to users when they desire addedfunctionality for their computers. Furthermore, the context servermodules may not only be supplied separately, but also together withappropriate input devices, such as the CS module 500 being distributedtogether with a GPS receiver that a user may connect to his or herwearable computer 120. Thus, a CS module may be provided to a user toprovide appropriate sets of attributes to the user, either alone or withappropriate input devices that may be required to generate theattributes under the CS. As another example, a self-contained biometricsensor, (e.g., a sensor sold under the brand name “LifeShirt” byVivoMetrics, Inc.), may be distributed with the cardiac condition CC242, and/or with body temperature, heart rate, blood pressure and bloodoxygen level context servers as noted above.

As yet another example, the user of the wearable computer 120 may wishto take up a new hobby such as scuba diving. As a result, the user maypurchase or receive a thematic set of attributes related to the theme“scuba diving,” as well as associated input devices. The user purchases,together as a package, a depth sensor, oxygen supply sensor, watercurrent direction/velocity sensor, as well as additional input devicesto replace existing devices (e.g., a video input device capable of beingused under water). In addition, associated context servers can bepurchased together with such input devices or separately, such as awater depth context server for producing a water depth attribute, anoxygen level context server for producing an oxygen level attribute, awater direction/velocity context server for producing a watercurrent/velocity attribute, and the like. The thematic set of attributesassociated with this underwater attribute set then includes the waterdepth, oxygen level, water direction/velocity and other attributes.

In addition to themes related to bodily activities (e.g., scuba divingand task performance) and to physical condition (e.g., location orcardiac condition), themes can also represent a mental or emotionalstate of a user, including a user's intentions or desires. For example,a cognitive load theme can include an attribute set whose individualattributes reflect context information such as whether the user istalking (and the volume of the speech), whether the user is talking onthe phone, physical movement such as walking or driving, whether theuser is stationary, whether the user is seated and stationary, ambientlight and sound, stress and hunger levels, a level of rest (e.g., a lowlevel due to a recent lack of sleep), current bodily activity such asreading e-mail or riding a bull, historical data (e.g., the user has alow threshold for cognitive load while watching baseball games), aphysical or mental disability, location (e.g., at home or therapist'soffice), the presence and frequency of user input such as keyboard ormouse activity, the presentation of output information to the user,emotional state, explicit indications from the user, etc. Indeed, thecognitive load attribute thematic set can contain numerous individualattributes which when aggregated provide a rich model of a user'scognitive load.

Another thematic attribute set employing numerous attributes to derive ahigher-level concept is a desired level of privacy theme or desiredscope of audience theme. The desired level of privacy/desired scope ofaudience themes may include individual attributes representing contextinformation such as the identity of others near the user, the proximityof others to the user, explicit tagging of activities or information(e.g., email in my personal account is private for only me, while emailin my family account is private for family members), the nature of workbeing performed (e.g., balancing a checkbook, playing a computer game,or revising a business spreadsheet), location, historical data, explicitindications from user, and other attributes.

As can be appreciated from the above examples, individual attributes ina thematic set of attributes are generated or derived in any number ofways. Those skilled in the relevant art will appreciate that the valuesfor the attributes can be stored in a variety of ways (e.g., a number ona scale of 1–100 or 0–255, a probability distribution, a value from adelimited set of possibilities, a fuzzy logic value, etc.). In addition,the above described detailed description of aspects of the invention maybe abstracted to a more general model, such as that shown in FIG. 7. Arepresentation of generating a thematic set of derived attributes basedon input data sources, where such derived attributes may be used tomodel characteristics of a user, is shown schematically as a system 700.Numerous input data sources 702 (1, 2, . . . N) are fed into severallogic modules 704 (1, 2, . . . N), which in turn produce respectivederived attributes 706 (1, 2, . . . N). For example, Input Data Sources1, representing measurements or signals from input devices, are fed intoa Logic Module 1 that includes rules for producing DerivedAttribute(s) 1. The Logic Module 1 may represent a CS or combined CS/CC.Likewise, a Logic Module 2 receives Input Data Sources 2 to produceDerived Attribute(s) 2. As shown by an arrow 708, the Input Data Sources2 also include input data from Input Data Sources 1. Thus, logic modulesmay share input data sources. As shown by an arrow 710, logic modulesmay also provide feedback to one another, such as a Derived Attribute 2being fed back as an input data source to the Logic Module N.

The logic modules 704 in the illustrated embodiment each process theinput data signals to produce derived attributes that could not bedirectly measured from the input data signals. Thus, such logic modulesmay provide intermediate or high-level attributes for modeling acharacteristic of a current state of the user. In other words, the logicmodules aggregate, interpret and provide attribute values that in turnmay be used to determine other characteristics of a user's situation orcontext. A thematic set of attributes based on a predetermined theme maybe those derived attributes from each of the logic modules 1, 2, . . .N. Moreover, a larger thematic set of attributes could include all ofthe derived attributes from all of the logic modules, (i.e., DerivedAttribute 1, Derived Attribute 2, . . . Derived Attribute N).

Importantly, each logic module may operate independently. For example,Logic Modules 1 and 2 produce at least Derived Attributes 1 and 2, andhave no need for coordinating with each other. Logic Module N, however,is dependent upon an input data source that is a derived attribute fromthe Logic Module 2. Thus, Logic Module N produces Derived Attributes Nthat cannot be independently measured or derived without the DerivedAttributes 2. For example, the Derived Attribute 2 may reflect theuser's cardiac condition, whereby the Derived Attribute N maycharacterize a user's mood based in part on his or her cardiaccondition.

Sets of attributes related to a common theme can be generated by one ormore of the logic modules 704. For example, one logic module may producea thematic set of attributes closely related to a given theme. The LogicModule 1, for example, may produce the location attribute set notedabove by performing the steps of module 400 of FIG. 4. Likewise, two ormore of the logic modules may together produce attributes related to acommon theme. Thus, the logic modules include various specialized setsof rules that are used to model particular aspects, environments orpersonas of the user and to produce corresponding sets of attributes.For example, sets of rules and corresponding sets of derived attributesmay be specialized based on occupation (e.g., a nurse, a secretary, afield technician, or a firefighter), gender (e.g., a woman's rules mayunderstand physiological symptoms related to pregnancy or otherfemale-specific conditions), age, or any of a variety of otherspecialization types.

A large number of themes can exist simultaneously or be usedalternately. In general, each thematic set of attributes shares a commontheme that characterizes or models a predetermined context of the user.What follows are examples of specialized themes for modelingcharacteristics of a user's current state.

Work—Attributes related to a work theme may include a location of theuser's office, logical and telecommunications links for the user'se-mail, voicemail, and telecommunications connections, links to relevantemployee handbook materials, office guidelines, policies, access codesfor accessing elevators, rooms, floors, and the like. The work attributeset could be supplied to each new employee by the employer (includingprocedures, forms, access keys, etc. that are particular to theemployer).

Entertainment—Entertainment attribute sets may include attributesrelated to a user's favorite web sites, television channels, musicpreferences, and the like. Attributes may be based on whether the useris participating or viewing, such as whether the user is participatingin a sport or just viewing the sport. If viewing, for example, then aset of attributes may include relevant statistics, biographies onplayers, location of a sporting event, location of adjacent parkinglocations and restaurants, and the like.

Errands—such as the grocery store example above.

Health—such as the cardiac condition attribute set above; attributes mayalso include emergency services and reflect ways in which a user'swearable PC may help the user provide or receive emergency services.

Safety—attributes may include OSHA regulations, escape routes,cautionary warnings (e.g., based on whether the computer detectshazards) and so forth to protect workers in a hostile environment.

Home including automatically adjusting environmental controls, selectingmusic and art, and so forth. A home attribute set may include attributesrelated to a desired ambient temperature, music volume, music selection,art genres to be displayed on high definition wall-mounted displaydevices, automatic door opening and unlocking features, access tohands-free telephone services such as voice mail and routing of incomingtelephone calls, and the like.

Children—for example, employing the location attribute set above toidentify whether a child can be authorized to be at a given location ata given time.

Place—such as the location CS and CCs explained above.

Law Enforcement—for example, employing the location attribute set aboveto apply parole requirements and ensure a person is at the rightlocation at the right time.

Medical Personnel—A medical personnel attribute set would includenumerous biometric attributes, particularly ones to be received frombiometric sensors not related to a wearable computer worn by a medicalservice provider. Other attributes include attributes related to apatient's name, age, relevant insurance data, medical history, drugreactions, and the like. A medical personnel attribute set would beuseful in distributed environments where wearable computers communicatewith one another; for example, an EMT with a computer approaching aperson may receive data wirelessly from the person's wearable computer,or a surgeon approaching an EMT may allow the EMT to communicateinformation about the person to the surgeon's wearable computer.

Routing—common couriers and retailers may use specialized sets ofattributes to facilitate package/product routing.

Military—Attributes for a soldier or group of soldiers may includerelevant biometric attributes, location attributes, output from a videoinput device, output from an audio input device or microphone,attributes regarding nearby soldiers, attributes regarding the soldier'smood, and the like. These place, health, and other attributes of asoldier may then be provided to regional headquarters, and ultimately tothe Pentagon under a large hierarchical structure.

Astronaut/Scuba Pack—As noted above, specific attributes for a space orunderwater environment include depth/altitude, pressure, oxygen levels,and the like.

Drivers/Pilots—Drivers or pilots of a car or plane or heavy equipmentwould expose certain attributes to the CM to allow a user to moreeffectively pilot or operate such equipment.

Mood—As noted above, attributes related to mood may include biometricinformation, such as the user's body temperature, galvanometricresponse, eye dilation attributes, brow furrowing sensor attributes, andthe like. Additional attributes reflecting a mood attribute set includeinformation reflecting a user's current activity (e.g., driving), speechpatterns and tone, whether additional people are present and who suchpeople are (e.g., ex-wife), and so forth.

Hostile Environment—Attributes related to a hostile environment includemany attribute sets noted above, including those for theastronauts/scuba pack, safety and location. Other attributes may includeattributes related to olfactory or chemical sensors that may detect thepresence of harmful chemicals in the air, a motion sensor attribute todetermine whether the user is being shaken, temperature sensors todetect ambient temperature, and the like.

Occupations—Attribute sets may differ for every human occupation. Acomputer programmer occupation attribute set may include numerousattributes related to logical data and telecommunications environment ofthe user, attributes related to database and numerous computer platformswith which the user may interact, and the like. Likewise, a patentattorney occupation attribute set may include attributes related to asmaller but different set of logical data and telecommunications links,including links to technical databases, previously drafted documents,patent office access attributes, as well as calendaring, contacts,docketing, word processing, and legal reference attributes.

A user's thematic set of attributes may well change as the user switchesroles or context. For example, the computer 120 can load and employ theuser's home attribute set when the user rises and prepares for work. Intransit, the computer 120 may employ a commuting attribute set, whichmay include links to previously identified news information links ordata feeds, together with cognitive load and user availabilityattributes. The commuting attribute set would then provide newsinformation that the user wishes to read or hear when the computerdetermines that the user's cognitive load and availability are suitable.Once the user arrives at work, the user's role changes, and the computerloads and employs the user's appropriate work or occupational attributeset. Context servers that supply attributes not relevant to theoccupational attribute set can be terminated. Of course, some attributesfrom the home and commuter attribute sets may still be relevant, andthus these context servers remain running (e.g., cognitive load orlocation attributes).

Under the above example, the computer 120 may switch themes from home tocommuter and from commuter to work based on user specifications.Alternatively, the computer may automatically switch between themesbased on other information, such as a time attribute. Thus, from 6:00 amto 8:00 am, the computer employs the home theme, but from 8:00 to 9:00and 9:00 to 6:00, the computer employs the commuter and occupationalthemes respectively. Under another embodiment, the computer may employanother thematic set of attributes to determine which theme to employ.For example, the computer may employ information from the location themewhen using the following rules: when located at home, employ the user'shome theme; when moving rapidly (e.g., above 5 mph), employ the commutertheme; and when located at work, employ the work theme. When thecomputer is unsure as to which theme to employ, the computer may ask theuser for input to select the desired attribute set. Of course, the usermay override an automatically selected attribute set if desired.

In addition to switching between thematic sets of attributes (andcorresponding context servers), the computer 120 may load and employ newthematic attributes while existing attribute sets remain active. Forexample, the user may travel to Guatemala to go scuba diving. Since theuser does not know how Spanish and is unfamiliar with the area, the usermay load a Guatemalan travel attribute set. The attribute set mayinclude context servers (which produce corresponding attributes) for thefollowing: translating Spanish language audio and text intocorresponding English audio and text attributes, providing Guatemalaetiquette attributes, hostile situation prediction attributes (ifpolitical tensions are high or the State Department has issued travelerwarnings), and the like. Additionally, context servers may be requiredto be modified in light of the location, such as context servers forproviding attributes based on the user's logical data andtelecommunications state where access to telecommunications differ fromthose in the user's home location. These new thematic attribute sets maybe loaded and employed with existing attribute sets, such as the scubadiving attribute set when the user is engaged in scuba diving inGuatemala.

The thematic attribute sets allow the user's wearable computer 120 tocharacterize the user's context and react accordingly. In otherembodiments, such context characterization may be performed elsewhere.For example, lower-level attribute and context characterization may beperformed by remote sensors, with such attribute sets provided to theuser's computer 120 that can in turn be provided or presented to the CMand context clients. This may be one way in which users may optimizetheir wearable computers. Other ways involve business orsituation-specific occasions where the wearable computer responds to andchanges the computer itself or the context characterizations that arepresented to the user. As an example, a previously created thematicattribute set with respect to a particular occupation (e.g., lawenforcement) can be loaded by the computer to readily provide moreaccurate contextual information to the user without human intervention.While a simpler context characterization system may ultimately correctlydetermine the user's context in a law enforcement environment, providinga specialized thematic attribute set for law enforcement significantlyshortcuts the computer's learning process.

Each thematic attribute set may also include relationships betweenattributes (e.g., inter-attribute logic) based on the theme. In oneembodiment, attributes may be selected from sets of predefinedattributes available for all context servers and clients. Suchpredefined sets of attributes allow a common meaning to be sharedbetween context clients and context servers for those attributes andtheir associated values, and can also allow a context client to requesta pre-defined attribute without having to determine whether theattribute has been created by a context server.

In one embodiment a plain-language, hierarchical, taxonometric attributenomenclature is used to name attributes, such as the example attributenomenclature illustrated in FIG. 8. The names within the nomenclatureare preferably specific so that there is no ambiguity as to what theyrepresent, and the ability to extend the nomenclature by adding newattribute names that conform to the hierarchical taxonomy of thenomenclature is preferably supported. The nomenclature preferably hasattribute names relating to a variety of aspects of the user. Forexample, as is illustrated in FIG. 8, the nomenclature preferably hasattribute names relating to the user's location, such asuser.location.latitude, user.location.building, anduser.location.street; the user's movement, such as user.speed anduser.direction; various user moods, such as user.mood.happiness,user.mood.anger, and user.mood.confusion; user activities, such asuser.activity.driving, user.activity.eating, and user.activity.sleeping;user physiology values, such as user.physiology.body_temperature anduser.physiology.blood_pressure; similar attributes of people other thanthe user, such as person.John_Smith.mood.happiness; attribute names foraspects of the computer system (or “platform”) and its user interface(or “UI”) capabilities, such asplatform.UI.oral_input_device_availability andplatform.UI.visual_output_device_availability; attribute names forattributes relating to the central processing unit (or “CPU”) of theplatform, such as platform.cpu.load and platform.cpu.speed; attributenames for aspects of the local environment, such asenvironment.local.temperature and environment.local.ambient_noise_level;attribute names for remote environments, such asenvironment.place.chicago.time andenvironment.place.san_diego.temperature; and attribute names relating tospecific applications (e.g., aspects of an email application), such asapplication.mail.available, application.mail.new_messages_waiting, andapplication.mail.messages_waiting_to_be_sent. In this manner, theattribute nomenclature provides effective names for attributes relatingto the user, the computer system, and the environment. Additionalattributes are illustrated in FIG. 8, while FIG. 9 illustrates analternate hierarchical taxonomy related to context, such that variousattributes could be added for each of the illustrated categories. Thoseskilled in the art will appreciate that for both FIG. 8 and FIG. 9,other categories and attributes could be added and existing categoriesand attributes could be removed or could have alternate names.

Based on the above, those skilled in the relevant art can appreciatethat separate thematic attribute sets may be created by selectingindividual attributes from the hierarchy presented in FIG. 8 or in FIG.9. For example, an occupational set of attributes may include:User.Location, User.Destination, and User.Proximity (selected from theUser attributes of FIG. 8), [Secretary].Location, [Secretary].Activityand [Secretary].Availability (selected from the Person attributes),Environment.Local.Time and Environment.Local.Date (selected from theEnvironment attributes) and Application.Mail and Application.Phone(selected from the Application attributes).

Furthermore, under another embodiment, thematic sets of attributes fallinto any of the following general classes:

-   1. Attributes related to the user's body and senses, including    visual, auditory, health and physiology;-   2. Attributes related to the user's feeling, including mood and    general temperament;-   3. Attributes related to the user's place and movement with respect    to places, including transportation, vehicles, physical environment,    home, office, and the like;-   4. Attributes relating to measurement and shape including size,    form, texture and the like;-   5. Attributes relating to living things and natural phenomenon, such    as plants, animals, weather and the like;-   6. Attributes relating to the user's behavior and will, including    veracity, motivation, impairment, skill and the like;-   7. Attributes relating to language, including means of    communication, language, diction, meaning and the like;-   8. Attributes relating to society and institutions, including    family, school, friendship, management and the like;-   9. Attributes relating to values and ideals, including right/wrong,    justice/injustice, religion, ethics and the like;-   10. Attributes relating to arts, including literature, motion    pictures, architecture and the like;-   11. Attributes relating to occupation and crafts, including    financial matters, user's vocation and avocations, and the like;-   12. Attributes related to sports and amusements, such as baseball,    skiing, card playing and the like; and-   13. Attributes related to science and technology, including the    user's computer, logical and telecommunications connectivity and    environment, physical qualities of materials, pure and applied    sciences and the like.

Many of the above classes of attributes reflect three states of a user'scontext: a state of the user, a state of the user's physicalsurroundings, and a state of the user's logical data andtelecommunications environment. The state of the user includes not onlythe user's physical and health condition, but also the user's mood,predisposition, preferences, memory and the like. The user's physicalsurroundings include the state of the user's computer, the user'sphysical location, the user's home, office, and common surroundings, andthe status of those surroundings. The user's logical data andtelecommunications environment (“virtual environment” or “cybersurroundings”) reflect connectivity between the user's PC and externalcomputing devices and/or data stored external to the user's wearablecomputer. For example, the user's logical data and telecommunicationsenvironment may represent IP addresses or ports to which the user'scomputer may couple or interface to exchange data with the externalworld, as well as to access the user's data such as from databases.

In a broad sense, aspects of the invention are directed to any dataprocessing system for use by a user. The data processing system includesat least first and second data sources, at least one data outputreceiver, and at least one processing unit with associated memory. Thedata sources provide respective first and second data signals, where thefirst and second data signals each reflect the state of the user, astate of the user's physical surroundings, or a state of the user'slogical data and telecommunications environment. The first and seconddata signals together represent a thematic data set characterizing theuser's context. The processing unit receives the first and second datasignals and executes at least one processing routine that receives thefirst and second data signals and provides an output signal to the dataoutput receiver, with the output signal representing a characteristic ofa current state of the user that cannot be directly measured from thefirst and second data signals. The processing unit may also executeanother processing routine that employs the first and second or otherdata signals to represent another characteristic of a current state ofthe user, with the two processes operating independently.

Those skilled in the relevant art will also appreciate that specializedversions of the body-mounted computer 120, characterization system 100,context servers and context client can be created for a variety ofpurposes. For example, embodiments of the system can be customized forparticular professions to better enable the system to automaticallycreate context models for members of those professions—in some suchsituations, the computer 120 may include a particular set of inputdevices and associated context servers for producing attributes for aparticular profession. Alternately, embodiments of the characterizationsystem can be customized for users' health problems, such as for peoplewith Alzheimer's disease, cardiac disease or pulmonary concerns. Thoseskilled in the relevant art will appreciate that a variety of suchphysiological conditions can be monitored, and that other specializedversions of the system can similarly be implemented.

Thus, as discussed above, themes can be used to represent a wide varietyof types of contextual aspects or situations, and often reflecthigh-level concepts of activities or states that may not be capturedwith individual attributes. In addition, information from and aboutcurrent themes can be used by a computing device (e.g., a wearablecomputer) to provide appropriate responses as the current contextchanges. In particular, in some embodiments information about one ormore current themes is used (e.g., by one or more executing CCs) topresent appropriate information and provide appropriate functionality toa user of the device.

In order to use information about a current theme to present appropriateinformation and functionality, it is first necessary to identify a themethat matches the current context. In an illustrative embodimentdiscussed below, each theme specifies a set of multiple attributes, andincludes for each of the specified attributes one or more possiblevalues for the attribute. Such a theme will match the current modeledcontext only if all the specified attributes have current values in themodeled context that are one of their possible values. In someembodiments, moreover, each specified attribute also includes anindication of whether it is required for the theme, and in suchembodiments the theme can still match the current context even if thecurrent values for non-required attributes do not match the specifiedpossible values. Those skilled in the art will appreciate that in someembodiments all specified attributes may be treated as being required,while in other embodiments attribute relevance can be specified in otherways (e.g., as optional, preferred, disfavored, with a numericalranking, etc.). In addition to specified attributes, each theme can alsospecify inter-attribute logic. Such logic can provide variousfunctionality, including allowing for more sophisticated determinationof whether a theme matches the current context by concurrently analyzingmultiple attributes in the specified set, such as by using boolean logic(e.g., (Attribute1[value]=“High” OR Attribute2[value]>10) AND NOTAttribute1[Uncertainty]=“High”) or other types of processing.

Those skilled in the art will appreciate that any changes in the modeledcontext can cause the current theme to no longer match the currentcontext and/or cause another theme that did not match the previousmodeled context to now match the current context. As such, the themethat is identified as the current theme can rapidly change based on thechanging context. If a change in the modeled context causes the previouscurrent theme to no longer match the current context and no other themesmatch the current context, various techniques can be used to continue toprovide appropriate responses to the user. In some embodiments theprevious current theme will continue to be treated as the current themeuntil another theme matches the current modeled context. In otherembodiments the user could be prompted, or other logic or information(e.g., default or preference information) could be used, to select anon-matching theme as a temporary current theme. In yet otherembodiments, additional themes may be loaded or retrieved in an attemptto make available a theme that matches the current modeled context.Alternately, no theme may be identified as the current theme untilchanges in the modeled context (or in one of the available themes)causes one of the available themes to match the context.

In the illustrative embodiment in which theme information is used topresent appropriate information and provide appropriate functionality,each theme has at least one associated theme layout that indicatesinformation and functionality appropriate to that theme. These themelayouts assist in presenting appropriate information and functionalitycontrols to a user. In particular, each theme layout can indicatespecific content or types of content appropriate for that theme, datalogic (e.g., an executable script) for processing content beforepresentation (e.g., to manipulate COM objects to retrieve data or toestablish a connection to an external database to allow retrieval ofdata), presentation logic (e.g., instructions or preferences) todetermine how the content should be presented to the user in a mannerconsistent with the theme, and functionality controls consistent withthe theme that allow interaction by the user with presented (andnon-presented) content. When a theme is selected as the current theme,one of the theme layouts associated with that theme can then be used toprovide to the user information and functionality appropriate to thecurrent theme.

In some embodiments, multiple themes can simultaneously match thecurrent context, and these matching themes are referred to as thecurrent theme set. For example, a user may both be “At Work” and “In AMeeting,” or “At Work” and “Talking On The Telephone.” Even though auser may simultaneously be involved with or part of multiple matchingthemes, each of the themes may have different types of appropriateinformation and functionality. For example, if the “At Work” theme isthe only current theme, the user may desire to see their work to-do listand indications of other co-workers who are nearby, while if the user istalking on the telephone the user may prefer to have access toinformation about the other people that are part of the conversation aswell as a functionality for taking notes. Thus, in order to determinewhat information and functionality is to be presented to the user at anytime, a mechanism is needed for selecting one or more appropriate themelayouts for presentation. In the illustrated embodiment, only a singletheme layout is presented at a time, and therefore the selectionmechanism first identifies one of the themes in the current theme set asbeing the current theme. After the current theme is identified, one ofthe theme layouts associated with that theme can then be selected as thecurrent theme layout for presentation to the user.

Those skilled in the art will appreciate that in other embodimentsmultiple theme layouts (whether they are associated with a singlecurrent theme or with multiple themes, such as the multiple themes inthe current theme set) could simultaneously be used to provideappropriate information and functionality to a user, such as indifferent windows on a display device, by using multiple output devices,by synthesizing some or all of the multiple theme layouts together intoa single presentation, etc. Those skilled in the art will alsoappreciate that even though one of the themes in the current theme setmay be identified as the current theme for purposes of determining whattheme layout to use, other types of processing may concurrently beoccurring (e.g., in the background) for other themes in the currenttheme set. For example, a safety theme could execute in the backgroundwhile a user is driving a car, and if a safety theme rule or other logicis triggered (e.g., by a road-traction attribute having its value changeto “slippery”) the safety theme could take appropriate action (e.g.,modify the value of a context attribute, override the current themelayout and present a warning message, notify an executing softwaremodule to log detailed context information, etc.).

The selection of a current theme from multiple matching themes in thecurrent theme set can be performed in a variety of ways. At a conceptuallevel, the user will typically desire to receive the information andfunctionality for whatever theme the user subjectively considers to bethe most important at that time (e.g., the primary focus of theirattention or interest, or the most theme conveying the most urgent orcritical information), and thus it is preferable to select that theme asthe current theme. In some situations the various themes may containinformation to facilitate the selection, such as a priority. If so, thehighest priority theme in the current theme set could be selected as thecurrent theme. In other situations, user-specific information could beused to select the current theme, such as by using user-specificpreference information (e.g., that is predefined by the user) or byallowing the user to explicitly select one of the themes in the currenttheme set (or to override the current theme set and select a theme thatdoes not match the current modeled context). Alternately, in someembodiments the themes in the current theme set could have an associateddegree of match (e.g., by using the presence or absence of non-requiredattributes for the theme), and the theme with the highest degree ofmatch could be selected. In yet other embodiments, themes could begrouped or associated with each other in various manners, and suchgroupings could be used to select the current theme. For example, atheme such as the “At Work” theme may be a relatively high-level generaltheme that has more-specific sub-themes (e.g., “Attending A Meeting” or“Writing A Memo”)—if so, the most-specific (or least-specific) matchingtheme could be selected as the current theme. Similarly, a general“Talking To Another Person” theme could have one or more more-specificsub-themes such as “Talking To Spouse” theme—those skilled in the artwill appreciate that the types of information of interest may differbased on the people, objects or places that are of current relevance.Alternately, themes could be categorized in various ways (e.g.,entertainment, convenience, productivity, and safety), and the currenttheme could be selected on the basis of belonging to a category ofcurrent highest importance or interest. Those skilled in the art willappreciate that a current theme could be selected in a variety of othersimilar ways.

In a similar manner, the selection of a current theme layout for thecurrent theme can be performed in a variety of ways. In someembodiments, each theme will have only one associated theme layout, andif so the single theme layout associated with the current theme isselected. In other embodiments different theme layouts for the sametheme could be specialized for different situations (e.g., optimized fordifferent types of output devices, different times of the day, week,month or year, different geographical locations, etc.), and if so themost appropriate theme layout for the current situation could beselected. In yet other situations one of the theme layouts associatedwith the current theme is a default theme layout, and if so it will beinitially selected as the current theme layout. Alternately, the usercould be prompted to indicate which theme layout to be used for thecurrent theme layout, either before or after any of the theme layoutsfor the theme have been presented. If user preference information isavailable (e.g., predefined information), such information could also beused in place of an explicit user indication.

Those skilled in the art will also appreciate that even if a singletheme layout is associated with the current theme, that theme layoutcould include logic or other information that allows it to optimized ormodified in various ways depending on the context at the time it ispresented. For example, the single theme layout could includeinformation to optimize the output for different output device types, orto change the information and/or functionality available based oninformation about the user (e.g., the user's demographic information,current occupation, a characterization of the user as a novice or anexpert with respect to the current theme, a current role in which theuser is acting such as a system administrator rather than an ordinaryuser, a current mode of the user such as work or a social environment,etc.).

FIG. 10 is a block diagram illustrating an embodiment of a computingdevice suitable for using theme and theme layout information to presentappropriate information and functionality to a user based on the currentcontext. In particular, a Thematic Response computing device 1000 isillustrated that can combine various context, content, theme, and userinformation in order to provide appropriate responses to the user basedon the current theme. In some embodiments, the Thematic Responsecomputing device may be a wearable personable computer that stores andexecutes each of the illustrated modules and data structures, while inother embodiments other computing device configurations are possible.

The computing device includes a CPU 1005, a storage 1020, memory 1030,and various I/O devices that include a network connection 1012, acomputer-readable media drive 1013, various input devices 1015 andvarious output devices 1017. The storage includes various data specificto the computing device and the device user (not shown), includingstored content 1021 for presentation. multiple user-specific themes 1023(e.g., themes that have been previously created by the user and/ordownloaded to the device and customized by the user), multiple themelayouts 1025 that are associated with the themes, various userpreference and customization data 1027, and various platform data 1029related to the computing device. In other embodiments that focus onusing theme-related information to provide appropriate responses to theuser other than the presentation of appropriate information, the themelayouts may be replaced by or supplemented with other types of themeresponse indicators that are associated with the various themes.

Various modules are executing in the memory, and they use various data(including the stored data) to provide appropriate information andfunctionality to a user of the computing device. In particular, aContext Modeler component 1031 is executing in memory in order to createand maintain a context model 1033 that describes the current context ofthe computing device and the device user. As previously discussed, theContext Modeler component can receive various context information 1060that is already in the form of context attributes, or it can insteadprocess received context information in order to convert it to contextattribute form, and then stores current values (and other associatedinformation) for the context attributes in the context model aspreviously discussed. In some embodiments, the Context Modeler componentcould be a collection of one or more CSes, or could instead be a CMreceiving such information from other CSes that are not shown. Thoseskilled in the art will appreciate that in other embodiments the ContextModeler component could be executing on another device and/or thecontext model could be stored on another if the computing device hasaccess to that other device.

A Context Chooser component 1032 is also executing in memory. TheContext Chooser component allows the user to explicitly set or modifyvarious types of context information, and makes corresponding changes tothe context model 1033. In so doing, the Context Chooser component canpresent appropriate information (e.g., current values for specifiedtypes of context information and interaction controls to allow valuemodification) to the user via one or more of the output devices, and canreceive instructions and new context information from the user via theinput devices. In some embodiments, there is a theme associated with theuser explicitly modifying the current context information, and the userinterface of the Context Chooser component is implemented via one ofmore theme layouts associated with that theme.

The memory also includes an executing Theme Modeler component 1040 thatrepeatedly identifies a current theme set having defined themes thatmatch the current context, selects a current theme, and then selects anappropriate type of response based on the current theme. In theillustrated embodiment that focuses on presentation of appropriateinformation and functionality, the selection of an appropriate type ofresponse includes selecting a current theme layout to be used inpresenting appropriate information and functionality to the user. Thoseskilled in the art will appreciate that in some embodiments the ThemeModeler component could be implemented as a CC.

The Theme Modeler component includes sub-components that includes aCurrent Theme Set Determiner 1045, a Current Theme Identifier 1043, anda Current Theme Layout Selector 1041. The Current Theme Set Determinercomponent receives information about the current context from thecontext model 1033 and about the defined themes 1023, and determines acurrent theme set by identifying the themes that match the currentcontext. In some embodiments, the Current Theme Set Determiner componentcreates or updates a current theme set data structure (not illustrated)in memory or on the storage.

The Current Theme Identifier component then receives information aboutthe themes in the current theme set, and selects a current theme that isintended to reflect the current or new focus of the user's attention. Insome embodiments, the Current Theme Identifier component will use userpreference data and/or platform data in order to assist in the selectionof the current theme. In some embodiments, the Current Theme Identifiercomponent will create or modify a current theme data structure (notillustrated) in memory or on the storage.

The Current Theme Layout Selector component next receives informationabout the current theme and about one or more theme layouts that areassociated with the current theme, and selects one of the theme layoutsto be used to present information to the user. In some embodiments, theCurrent Theme Layout Selector will use user preference data and/orplatform data in order to select the current theme layout. In addition,in some embodiments the Current Theme Layout Selector component willcreate or maintain a current theme layout data structure (notillustrated) in memory or on the storage.

A Theme Layout Presenter component 1035 is also executing in memory, andit is responsible for the presentation of appropriate information andfunctionality to the user. As with the Theme Modeler component, theTheme Layout Presenter component could be implemented in someembodiments as a CC. The Theme Layout Presenter component receivesinformation about the current theme layout, and processes that themelayout to generate an appropriate presentation for the user. Inparticular, the Theme Layout Presenter component will gather variouscontent that is specified by the current theme layout, whether it isstored as part of the current theme layout, stored separately as content1021 on the computing device, or stored externally as various content1070. The Theme Layout Presenter component can also receive and use userpreference and/or customization data, such as to adjust the presentationfor user preferences (e.g., to use a specified a appearance scheme, orto use specific types of user interaction controls) or to customize adefault theme layout in various ways. In a similar manner, the ThemeLayout Presenter component can also receive and use platform data toadjust the presentation, such as to determine available and/or optimaloutput devices, or to optimize the information and interactions to bepresented based on the output devices to be used. While not shown in theillustrated embodiment, the Theme Layout Presenter component could alsoin some embodiments receive and use various context information from thecontext model.

After gathering the appropriate information and processing it as needed,the Theme Layout Presenter component then presents the appropriateinformation and interaction controls on the appropriate output devices.In some embodiments, the Theme Layout Presenter component will alsoselectively control various input devices, such as to enable or disablesuch devices depending on the available interaction controls andfunctionality of the current presentation. In addition, in someembodiments information received from the user via the input devices canbe provided to and used by Theme Layout Presenter component in order tomodify the current theme layout presentation, or could similarly beprovided to and used by the Context Modeler component, Context Choosercomponent, and/or Theme Modeler component. Those skilled in the art willappreciate that in other embodiments that focus on providing appropriateresponses based on the current theme that involve responses other thanpresenting appropriate information, the Theme Layout Presenter componentcould be replaced by or supplemented with a more general Theme ResponseGenerator component.

In the illustrated embodiment, the Thematic Response computing device isalso communicating with other computing devices 1050 via a network 1080.Those skilled in the art will appreciate that in other embodiments theThematic Response computing device may execute as a self-containedstand-alone device. Conversely, in other embodiments some or all of thestored data or executing components may be located on one or more of theother computing devices, with the data or execution results provided tothe Thematic Response computing device as needed. For example, in someembodiments the Thematic Response computing device could be a thinclient that had little or no computing or storage capabilities, insteadacting merely as a input device and/or an output presentation device forother computing devices. In addition, those skilled in the art willappreciate that the Thematic Response computing device can obtaincontent and context information from a variety of external sources,including one or more of the computing devices 1050 and other externaldata sources (not shown).

Those skilled in the art will appreciate that computing devices 1000 and1050 are merely illustrative and are not intended to limit the scope ofthe present invention. Computer system 1000 may be connected to otherdevices that are not illustrated, including through one or more networkssuch as the Internet or via the World Wide Web (WWW). In addition, thefunctionality provided by the illustrated components may in someembodiments be combined in fewer components or distributed in additionalcomponents. Similarly, in some embodiments the functionality of some ofthe illustrated components may not be provided and/or other additionalfunctionality may be available. For example, in some embodimentsautomated context modeling functionality may not be provided, while inother embodiments additional functionality may be available to allow auser to create, modify, customize, distribute, retrieve, purchase, sell,and otherwise use themes and theme-related information.

Those skilled in the art will also appreciate that, while variouscomponents and the context model are illustrated as being stored inmemory while being used, these items or portions of them can betransferred between memory and other storage devices for purposes ofmemory management and data integrity. Similarly, data illustrated asbeing present on storage while being used can instead be present inmemory and transferred between storage and memory. Alternately, in otherembodiments some or all of the software modules may execute in memory onanother device and communicate with the computing device 1000 viainter-computer communication. Some or all of the components or datastructures may also be stored (e.g., as instructions or structured data)on a computer-readable medium, such as a hard disk, a memory, a network,or a portable article to be read by an appropriate drive. The componentsand data structures can also be transmitted as generated data signals(e.g., as part of a carrier wave) on a variety of computer-readabletransmission mediums, including wireless-based and wired/cable-basedmediums.

In addition, a computing device or computer may comprise any combinationof hardware or software that can provide storage, provide processing,provide input or output capability, and/or interact with other devicesthat provide such capabilities, including computers, network devices,internet appliances, PDAs, wireless phones, pagers, electronicorganizers, television-based systems and various other consumer productsthat include inter-communication capabilities. Accordingly, the presentinvention may be practiced with other computer system configurations.

FIGS. 11A–11L provide various examples of changing theme layoutpresentations based on changes to a current context, and FIGS. 11M–11Oprovide examples of explicit user control of various theme-relatedinformation. In the theme layout presentations illustrated, only thevisual aspects of the theme layout presentation are illustrated—thoseskilled in the art will appreciate that other types of output devicesand user senses could be used either concurrently or alternatively. Inaddition, in the illustrated embodiment the theme layout presentation isperformed by a wearable personal computer that remains with the userthroughout a day. In most situations the illustrated theme layoutpresentation will fill a display screen available on the wearablecomputer, although those skilled in the art will appreciate that thewearable personal computer could also opportunistically use otherdisplay devices as they become available and that other informationcould additionally be displayed.

With respect to FIG. 11A, a wearable computing device user has recentlyrisen, and the wearable determines that the most appropriate currenttheme layout is one related to a current “Starting The Day” theme. As isshown, the theme layout presentation includes a variety of differenttypes of content sets, such as news headlines 1105, a current to-do list1110, and a schedule/calendar 1115. Such content sets can be gatheredfrom a variety of sources, such as the news headlines being retrievedfrom an external source (e.g., a news organization's web site orsubscription service), the to-do list and calendar being retrieved fromthe user's personal or work data store, or context attribute informationsuch as the inside temperature 1109 or indications of people at homebeing retrieved from an appropriate context model or context informationsupplier. Those skilled in the art will appreciate that the theme layoutcan specify the manner in which to gather the information in a varietyof ways, such as a URL, a database query, executable logic such as ascript, etc.

In addition to presenting various information, various functionality andinteraction controls are also presented to the user. For example, as isshown in the news headline section of the theme layout presentation, ascroll bar interaction control is available, and other non-visibleinteraction controls (e.g., a pop-up menu based on a user indication orcursor position) may also be available. In addition, some or all of thepresented information may itself be selectable by the user, such as toselect a new theme and present corresponding information. For example,each of the headlines may be selectable by the user, and if the userselects a headline such as headline 1107 the current theme could switchto a “Reading News” theme whose associated theme layout presentationwould include the corresponding news story. Similarly, the user couldselect an item on the to-do list such as task 1112 in order to bepresented with more detailed information about the task or to modify thetask (e.g., to change its priority or indicate that it is completed).Similarly, the user could select a person indication such as “The Wife”1117 to obtain information about that person (e.g., their current statuswith respect to getting ready).

The current theme layout presentation also illustrates that presentedinformation can be specific to the user (e.g., the to-do list), specificto a group or other category to which the user belongs (e.g., thecalendar information if it is retrieved from a company-wide workcalendar rather than the user's personal calendar), or independent ofthe user (e.g., current world headlines). In most embodiments, if the“Starting The Day” theme and its associated theme layout weredistributed to another user, user-specific or group-specific data willnot accompany the theme and theme layout. For example, the presentationof this same theme layout for a different user may have the same typesof presented information and may include the same current worldwideheadlines, but the actual data presented in the other sections of thepresentation would be modified to reflect the new user (e.g., retrievinga value for the inside temperature attribute 1109 from a context modelfor the new user, and retrieving to-do list and calendar information forthat new user). While not illustrated in the current theme layoutpresentation, those skilled in the art will also appreciate that sometheme layout presentations can include static information stored withthe theme or theme layout, such as a smiley face logo to be displayed ora indication of lateness that is displayed if the current time exceedssome defined deadline information.

After the user gets ready, they begin to drive to work, and the wearablepersonal computer accordingly modifies the current theme layoutpresentation to that illustrated in FIG. 11B. As is shown, a variety oftypes of information can be displayed, such as a text description of thedestination 1120, a local map 1121, text driving directions 1122, anupdated traffic congestion map 1123, schedule information 1125, andtask-related information 1127. As with other theme layout presentations,the data to be presented can be retrieved in various ways, such asinteractively from external data stores or instead from a local datastore. In addition, those skilled in the art will appreciate thatdifferent theme layout presentations can share information with othertheme layout presentations, such as schedule information being availablein this and the previously illustrated theme layout presentation.

FIG. 11C illustrates that as the user arrives at work a new theme layoutpresentation will be used that includes information more relevant to theworkplace than to driving, such as an indication of people in thebuilding. This theme layout presentation also illustrates that data forpresentation can be gathered by interacting with other context models,such as determining the people that are currently in the building orretrieving work bulletin board information by querying a context modelor other data store for the workplace. When retrieving such information,those skilled in the art will appreciate that access controls orsecurity measures can be used as appropriate in order to protect privateor sensitive information. After the user enters the workplace, thecurrent theme layout presentation switches to a generic “At Work” themelayout that is illustrated in FIG. 11D. This theme layout presentationincludes calendar information, task list information, and indications ofnearby people.

FIG. 11E illustrates a different type of theme layout presentation thatis specific to a particular person. In particular, the user's wearablecomputing device may have determined that the user has begun to talk toone of the nearby people, Bill Watson. Alternately, the system may havean important reminder related to Bill, and thus modified the display toprovide that information to the user. Alternately, the user may haveinteractively selected the displayed indication 1130 of Bill Watson inFIG. 11D. Regardless of the manner in which this Person theme wasselected, the corresponding theme layout presentation can includevarious relevant information such as a picture of the person, contactinformation about them, and reminders related to the person. In thecurrent situation, a generic person theme layout is currently beingused, with Bill Watson being selected as the current topic or focus. Asillustrated by interaction control 1135, the user could interactivelyselect a different user, and if so the sections of the theme layoutpresentation would remain the same but the specific informationdisplayed would change to correspond to the newly selected person. Forother people or in other embodiments, more specific theme layoutpresentations could be used. For example, if the topic of the persontheme was instead the user's boss or the CEO of the company, the usermay have a theme layout that is specific to that person and thatincludes other types of relevant information or functionality.Similarly, if the topic of the person theme was a member of the user'swork group, a theme layout specific to that group could be used. If thesystem determines that it is no longer appropriate to display the Persontheme layout (whether automatically or via an explicit user indication),the system could then return to the general At Work theme layoutpresentation previously illustrated in FIG. 11D.

FIG. 11F illustrates a more specific work theme layout presentation thatis relevant when the user is attending a meeting. In addition toindicating people present in the meeting, the theme layout presentationcan include a section for previously recorded notes and/or new notes tobe generated during the meeting. In addition, the illustrated meetingtheme layout presentation includes information about things to mentionat the meeting, some of which may be standard for each status meetingand others may be specific to the current meeting. FIG. 11G illustratesanother person theme that has Dan Newell as the topic. Information abouta person can be displayed in various situations, such as if the systemautomatically detects that the person is speaking in the meeting or thatthe person has become the focus of the user's attention (e.g., by theuser looking at them for a sustained period or by the user explicitlyindicating that person). FIGS. 11H and 11I illustrate other theme layoutpresentations that may be used throughout the day, such as a themelayout presentation related to going to lunch or to a specific task suchas repairing a computer. As with the previously discussed Person theme,the computer repair theme layout presentation illustrated in FIG. 11I isan object-specific theme in which a particular object (e.g., “Josh'sComputer”) is currently selected as the topic of the theme and the themelayout correspondingly displays information related to that object.

FIG. 11J illustrates a theme layout presentation for a “Driving Home”theme that is similar to the “Driving To Work” theme previouslydiscussed. In some embodiments, a generic Driving To Location themecould be used in which a specific location was the topic of the theme.However, in the illustrated embodiment it is appropriate to presentdifferent types of information and controls to the user when they aredriving home than were appropriate when the user was driving to work.For example, the illustrated Driving Home theme layout presentationincludes current information about the status of specific people such asthe user's wife and objects such as the user's house. In addition, theillustrated theme layout presentation includes interaction controls 1145to provide currently appropriate functionality, such as to turn on theheat at home or to call the user's wife.

FIG. 11K illustrates the presentation of information related to a safetytheme layout. In particular, a “Safety” theme has been executing in thebackground throughout the day (e.g., by being designed to always matchany context and thus be part of every current theme set), and logicassociated with the Safety theme (e.g., a rule) has detected a suddensafety danger. In the illustrated embodiment, the wearable computingdevice has received information that traffic will abruptly slow or stop.Since the Safety theme has precedence over the Driving Home theme (e.g.,has a higher priority), the Safety theme takes over the display andissues a warning message 1190. In the illustrated embodiment, the Safetytheme layout presentation does not completely replace the previous themelayout presentation, but instead simultaneously presents information.Those skilled in the art will appreciate that in other embodiments theSafety theme layout presentation would completely replace the previouslypresented theme layout. In addition, the visual aspect of the Safetytheme layout presentation may also be accompanied by other non-visiblepresentation, such as an auditory alert and/or a haptic notification.

FIG. 11L illustrates a theme layout presentation related to the“Arriving Home” theme. Those skilled in the art will appreciate that thetheme layout presentations illustrated in FIGS. 11A–11L are provided forillustrative purposes only, and are not intended to limit the scope ofthe invention.

FIGS. 11M–11O provide examples of user interfaces with which a user canexplicitly select or modify theme-related information. In particular,FIG. 11M provides an example user interface with which a user can selecta new current theme. In particular, a variety of possible themes 1150are illustrated, with the current theme visually indicated in thisexample embodiment in bold. Those skilled in the art will appreciatethat in some embodiments any defined themes may be made available forselection (including themes not currently loaded, such as themesaccessible from a local or remote storage location), while in otherembodiments the themes displayed may be limited in various fashions(e.g., to themes in the current theme set). In addition, the illustratedembodiment demonstrates that displayed themes can also be grouped orcategorized in various ways, such as by including a work-related groupof themes 1155 that includes multiple themes. If a user selects a newcurrent theme, the system can modify the current display to reflect atheme layout associated with the new current theme, such as after theuser indicates a final selection with the “OK” interaction control 1157.Alternately, the user in the illustrated embodiment can use the “Cancel”interaction control 1157 to retain the previously selected currenttheme.

In a similar manner, FIG. 11N illustrates an example user interface toallow a user to explicitly modify the themes that are part of thecurrent theme set. As with FIG. 11M, the user interface in FIG. 11Nillustrates multiple themes 1150, but the illustrated themes are limitedto those that are currently members of the current theme set. Inaddition, interaction controls 1157 allow the user to add or remove atheme from the current theme set. Selecting the “Add Theme” interactioncontrol may, for example, prompt a display such as is shown for FIG. 11Min which one or more themes that are not currently present in thecurrent theme set can be selected. Those skilled in the art willappreciate that other interaction controls could also be present, suchas a control to temporarily suspend the execution of a theme that ispart of the current theme set without removing it or to resume theexecution of a suspended theme. Those skilled in the art will appreciatethat the addition of new themes to the current theme set can result inthemes that do not match the current context being executed in thebackground, but processing or other functionality associated with thosethemes (discussed in greater detail below) may nonetheless be currentlyuseful.

FIG. 11O provides an example user interface with which a user can selecta new theme layout to be the current theme layout, thus explicitlymodifying the current presentation of information and functionality. Aspreviously discussed, users can also explicitly select new current themelayouts in other manners, such as by selecting information related to aperson, place, or object that is presented as part of another themelayout presentation. In the illustrated embodiment, multiple themelayouts for the current theme are displayed for selection by the user,with the current theme layout visually indicated in a bold manner. Ifthe user selects a different theme layout and selects the “OK”interaction control 1157, the system will modify the presentation ofinformation and functionality to the user to reflect the new currenttheme layout. Those skilled in the art will appreciate that in otherembodiments theme layouts associated with multiple different themescould be simultaneously displayed to the user for selection.

FIGS. 12A-12H provide an alternative method for the user to explicitlyprovide information or instructions. In particular, these Figuresprovide examples of a user interface for a user to explicitly specifycontext information about themselves, such as via a Context Choosermodule. As the user makes changes to the current modeled context, thesechanges can propagate through the theme mechanism and cause changes inthe current theme set, current theme, and/or current theme layout.

FIG. 12A illustrates a user interface with which a variety of types ofcontextual information can be inspected, configured and modified. Theuser interface includes a variety of choices 1210 that the user canselect, such as choice 1 to create a new theme, choice 2 to organizeexisting themes, and choice 3 to set the current theme. Other displayedchoices, including choices 4–9, correspond to categories of contextualinformation, and multiple tabs 1220 are also displayed that eachcorrespond to one of the categories. In some embodiments, users canchange the categories of contextual information that are displayedeither by selecting an appropriate one of the choices 1210 or byselecting the appropriate tab 1220.

In the illustrated embodiment, a Me choice 1215 that corresponds toinformation about the user has been selected, with the tab 1225correspondingly being the visible tab. As a result, a variety ofuser-related information is displayed in the right portion of the userinterface. As is shown, the user interface allows the user to specify avariety of types of information about themselves, such as the user'smood or health status using interaction controls 1235. Similarly, theuser can have one or more defined modes that may affect multiple contextattributes, user preferences, and/or computer devices. In theillustrated embodiment, the user can specify a “Solitude” mode using aninteraction control 1235. In addition, various user sense indicators1230 are indicated, such as to provide status information to the userabout senses to which the computer may currently present informationand/or to allow the user to control which input/output devices arecurrently in use. The user interface also includes controls 1237 thataffect interaction between the user and other users, such as to allowthe user to specify whether some or all of their context information isavailable to others or to view context information for another user. Inthe illustrated embodiment, after the user has finished inspectingand/or modifying context information about themselves, the user canselect a different category of contextual information, such as byhighlighting the Activity choice 1217.

FIG. 12B illustrates the user interface after the user has selected theActivity choice to display Activity-related contextual information. Aswould be expected, the Activity-related tab 1227 is now the currentlyvisible tab, and a variety of types of Activity-related contextualinformation is displayed. In the illustrated embodiment, the user canspecify information related to a current activity, a current mode, acurrent theme, and various project-specific information.

FIG. 12C illustrates the user interface displaying a variety of contextinformation related to the “People” category, as is reflected by theselection of the People tab 1241. As is shown, in the illustratedembodiment information is presented only about people that are currentlyaround the user—those skilled in the art will appreciate, however, thatin other embodiments information about other types of people could alsobe indicated (e.g., people of particular interest, such as a spouse, orpeople of current interest, such as other people with whom the user isconversing over a remote communications mechanism). In the illustrateduser interface embodiment, the user can use interaction controls to addor remove people that are considered to be around the user, as well asto specify one or more of the listed people as a current focus of auser's attention (e.g., to be used as the topic of the Person theme if aPerson theme layout is presented).

In a similar manner, FIGS. 12D–12H provide examples of the userinterface displaying other categories of information. FIG. 12D displaysvarious location information about the user, including their currentlocation and intended destination. While not illustrated in the currentembodiment, other embodiments could include an interaction control toallow the user to specify whether to share this particular type ofcontext information (or other types of context information) with others,and if so with whom. FIG. 12E displays environment-related contextualinformation, such as information about the temperature, light, andsound. FIG. 12F displays contextual information related to objects orconceptual things that are around the user, such as computing devices.As with the previously discussed People-related contextual information,the user can specify a current focus of attention among those displayedobjects, and those skilled in the art will appreciate that in anotherembodiments information about objects other than those near the usercould also be displayed. FIG. 12G displays contextual informationrelated to the computing system itself, such as information about thebattery, I/O devices, installed software or networking mechanisms, etc.FIG. 12H provides “About” information related to the software executingthe user interface. Those skilled in the art will appreciate that avariety of other types of contextual information could similarly bedisplayed and modified via such an interface, and that a variety ofother types of user interfaces could similarly be used to display and/ormodify contextual information.

FIG. 13 provides an example of a theme data structure 1300. The datastructure includes a variety of properties 1302 and corresponding values1304. Those skilled in the art will appreciate that the details of theillustrated data structures are for illustrative purposes only, and arenot intended to limit the scope of the invention.

In the illustrative example, the theme data structure includesproperties as follows: Name, Description, Security, Permission, Privacy,Priority, Group-ID, Time-Active, Source, Theme-Content, New-Attributes,Theme-Matching, Theme-Logic, Attribute-Set, etc. In the illustrativeembodiment, a data structure for the “Driving To Work” theme is shown.The theme includes a textual description, such as for use whendisplaying the theme to the user for modification, a theme priority thatcan be used for selecting a current theme from multiple themes in thecurrent theme set, and a categorization of the theme as belonging to agroup of themes with an ID of 23. The illustrated theme also includes anindication of when the theme is active such that it is allowed to matchthe current context, with this theme active on weekdays from 6 a.m. to11 a.m. The theme also includes an indication of the source thatsupplied the theme, such as a third-party source from whom the theme waspurchased. The theme includes an indication of theme-specific contentthat is stored locally, which in the illustrated embodiment is a logo ofthe company that sold the theme. The theme also includes an indicationof new theme-specific attributes that are used as part of this theme,along with an indication of a source from whom values for the attributecan be received. In some embodiments, instructions on how to locateand/or load the theme-specific attribute or theme-specific CS could alsobe included.

The theme-matching property of the theme provides one or more sets ofattributes and corresponding values to allow the system to determinewhether the theme matches the current context. In the illustratedembodiment, each attribute-value pair also includes an indication ofwhether that attribute-value pair is required to match the currentcontext. Theme logic is also associated with the theme data structure,including an instruction to modify the access that is allowed to anattribute (i.e., the user.location attribute) while the theme is thecurrent theme and/or in the current theme set. Other theme logicincludes checking the values of standard attributes or theme-specificattributes in order to perform various types of actions. As previouslynoted, in some embodiments theme logic can also be used to determinewhether the theme matches the current context. Those skilled in the artwill appreciate that in other embodiments theme logic information (orother information) may not be directed stored in the data structure,such as the theme data structure instead including a link to one or moreexecutable programs that implement the theme logic. The theme datastructure also includes links to each of the attributes in the attributeset for the theme, with attribute data structures 1390 illustrated.

The illustrated theme data structure also includes Privacy, Security,and Permission information. In the illustrated embodiment, the Privacyinformation can affect capabilities such as where data generated whilethe theme is active is stored and who has access to it. In this manner,work-related themes can ensure that data generated while that theme iscurrent will be stored in an accessible work location, and non-workthemes can ensure that generated data will remain private. Typically,individual themes share the same context modeling data store. However,groups of themes may have distinct databases. Themes that share the samepreferred privacy indication would tend to access the same database.Examples of different privacy values/schemes include the following:Private, Public, Work, Family, Friends, Acquaintances, People inimmediate vicinity, and Everyone in my contact list. These schemes arenot necessarily mutually exclusive (e.g., Family+Friends), though theycan be (e.g., private vs. public). They can be combined in someembodiments with Boolean or other logic. Example themes which a usermight specify as private include Driving Home, At Home, Talking on MyPersonal Mobile Phone, Receiving Medical Treatment, etc.

The illustrated theme data structure also includes Security andPermission information. In the illustrated embodiment, Permissioninformation is used to specify what types of activities different userscan engage in with respect to the theme, and Security information can bespecified to control the various types of access. In the illustratedembodiment, similar groups as those mentioned above can be used whenspecifying permission or security information. In the illustratedembodiment, access to information about the scheme (including whetherthe theme is executing or is the current theme) is available to thecurrent user, to other users that are part of the current user's Familyor Friends groups of users, and to the Source of the theme. Access bythe Source allows them to monitor usage of the theme. In Permissioninformation also specifies that only the current user can modify thetheme, and that no one is allowed to copy the theme. A variety of othertypes of permission controls could similarly be specified. In addition,controls such as access or modifiability could be limited to a subset ofthe theme in other embodiments. Similarly, various restrictions on suchaccess could be specified, such as “Show my current location toco-workers only when I am in route to the office or located within 100yards of it.”

The user does not necessarily determine some or all of the activepermission scheme. For example, an employer may distribute a theme toemployees that it does not want modifiable (e.g., a safety theme).Alternately, in other institutional applications (e.g., a hospital orthe military), the control of the content, how the context is beingmodeled, how the context information is being shared, and any othersystem controlled activity can be controlled by a remote and/orcentralized authority. Remote control does not require continuouscommunication—the theme (or a theme-specific CS) could have a “dead air”rule such that it ceases to work if the system does not receivecontinuous authorization for use. Permission information could alsoinclude information related to distribution of the theme, such as amechanism for charging a user and supplying the payment to a specifiedsource when the theme is distributed.

In the illustrated theme data structure, the theme logic indicates anability to specify access to a context attribute, such as to provide orlimit access to information to other users. Similarly, themes can alsospecify other types of information, such as a data store from which datacan be retrieved or to which data can be stored. Themes can also setattribute values, such as to ensure that a privacy attribute has anappropriate value. In some situations the values will be set onlytemporarily while the theme is current (or is in the current theme set),and will return to their previous value or to a default value when thetheme is no longer current (or is not in the current theme set).Alternately, in other situations a theme can permanently set anattribute value such that the value remains even when the theme is nolonger current.

In addition to setting context information, themes can also specifyother types of information, such as whether some or all of theinformation about the theme is available to other themes. In somesituations a theme may specify that even the existence of the theme orthe fact that the theme is in the current theme set or is the currenttheme is not available to other themes, although such a specificationmay also be overridden by the system if necessary (e.g., making suchinformation available to a safety theme that needs to override thedisplay to present an important message). When information about otherthemes is available, a theme (e.g., the current theme) or theme layoutcan use such information as part of the theme logic to modify thepresentation of the current theme layout. In other embodiments themescould included various functionality to allow them to interact, such asfor all of the themes in the current theme set to cooperatively selectone of the themes as the current theme rather than having this performedby a separate executable module.

While the illustrated theme indicates that it is a member of a group, avariety of other types of group and hierarchical information could bespecified in other embodiments. In some embodiments themes can bespecified in a hierarchical arrangement such that themes at lower levelsof the hierarchy could inherit default attributes or functionality fromhigher-level themes. For example, a “Talking To Spouse” theme could bespecified as being a descendant or child of a “Talking To Person” theme,with the child theme including only the information or logic that isdifferent than the parent theme. In addition to inheriting informationand functionality, such hierarchical arrangements can be used in somesituations to prioritize themes. Moreover, in some situations themesand/or theme layouts could be constructed in a hierarchical manner byusing other themes or theme layouts. For example, with a theme layoutsuch as is illustrated in FIG. 11B, the theme layout may be constructedby combining other theme layouts for text map directions and trafficcondition information with other information specific to the Driving ToWork theme.

In addition, groups of themes can be specified in various ways and usedfor various purposes (e.g., as an organizational scheme when presentinga list of themes to a user). As previously noted, group membership couldbe used to specify various common properties, like Privacy, Security,Permission, Priority, Time-Active, Theme-Content and/or otherproperties. In this way, a user could know what other themes could beselected that would not change the current settings for such properties.Alternately, an institutional organization could use group membership tocontrol access to and functionality associated with different groups—forexample, a retailer can maintain distinct groups of themes foremployees, vendors, contractors, identified or anonymous shoppers.

Themes can also be categorized in various ways, and themes in someembodiments could store related categorical information. For example, insome embodiments themes could be categorized as “Situational Themes”that describe the intent of the user specific to that person's activity(e.g., driving), “Person Themes” containing information related topeople, “Object Themes” containing object-related information, or “PlaceThemes” containing place-related information. In other embodiments,themes could be categorized into high-level categories such as“Intent/Content,” “Ability,” “Platform,” and “Personal Preference.”Intent/Content themes could include sub-categories such as “Work,”“Chores,” “Entertainment,” etc. Each of these could include varioussub-categories. Ability themes could include sub-categories for themessuch as “Bodily Activity,” “Available Attention,” etc. An importantbenefit of modeling a user's body activity is to determine how well theycan receive or generate information. For example, if the user iswalking, the system can modify the UI such that the fine motor controlrequired for two-dimensional cursor control (e.g., with a mouse ortrackball) is not required, and could instead use UI elements thatpresent choices with one-dimensional cursor control (e.g., with a scrollwheel or voice commands). Example types of relevant activities could beWalking, Sitting With A Horizontal Work Surface, Driving, etc. Attentioncould include an indication of a user's preferred data complexity leveland their current cognitive load. Platform themes could be used toprovide information about what to do when system capability changesconfiguration. For example, if the device battery is being consumed at arate higher than can be sustained until historical battery chargingresources are available (accounting for route and rate of travel), themelogic could initiate power saving rules. Alternately, if a preferredoutput device is not available, theme logic could determine and suggestthe best alternatives. Personal Preference themes can includesub-categories related to the device UI, Privacy, Solitude, Safety,Convenience, etc. UI-related themes could provide functionality such asincreasing the audio volume when the ambient noise increases, orincreasing contrast when the ambient light increases. A Privacy themecould include detailed information that is shared with other themes. ThePrivacy theme could have an associated theme layout that would presentsettings for various privacy types, and/or it could present a summary ofprivacy (e.g., a single word like “private” or “public”) to other themesor on a displayed system status tool bar. Information that is explicitlyset in such a Privacy theme could also override a default settingcontained in a specific theme without modifying the default (so that thedefault setting could later be used if the explicit Privacy themesetting were removed). A Solitude theme could indicate not to interruptthe user unless it is an emergency. A Safety theme could begin recordingall sensors and attributes at high data rates if the system infers thatI am having a health crises (e.g., car is skidding at high speed or mypulse is erratic). The system could then indicate to the user that asafety issue has been determined, and alert emergency personnel andfamily if attempts to get acknowledgement or correction from the user isnot received. Convenience themes could provide functionality such ascollecting suggestions from family members for errands to perform beforecoming home from work. Such an errand list could be shared by otherfamily members, and an indication provided if an errand is accepted byfamily member.

Those skilled in the art will appreciate that themes can be grouped,categorized and hierarchically structured in a variety of other ways,some of which are described above.

FIG. 14 is a block diagram illustrating an embodiment of a Theme Servercomputing device suitable for distributing themes and theme-relatedinformation to various Theme User computing devices that will modify anduse those and other themes. In particular, a Theme Server computingdevice 1400 is illustrated that includes a CPU 1405, a storage 1420,memory 1430, and various I/O devices 1410.

The storage includes various themes and theme-related information fordistribution to user computing devices, including multiple themes 1421.Depending on the themes for distribution, the storage can also includemultiple theme layouts 1422 that are each associated with one of thethemes, various theme-specific attributes 1423 that are part of the setof thematic attributes for at least one of the themes (e.g., by beingused in theme-matching or theme-logic), various theme CSes 1424 able togenerate values for one or more of the theme attributes, and varioustheme CCs 1426 able to use information from at least one of the themes(e.g., theme attribute values) to generate appropriate responses for auser that is part of that theme. The storage can also optionally includetheme categorization or grouping information 1428, such as fordetermining associated themes that may be distributed together, and canoptionally include user or group categorization information 1429, suchas for categorizing users and groups of users in order to determine whatthemes are appropriate for those users.

Various modules are executing in the memory in order to create,distribute, and monitor the use of distributed themes and theme-relatedinformation. In particular, a Theme Creator/Modifier component 1436 isexecuting in memory so that a user can use the component to create ormodify themes and/or theme-related information, such as the themes andtheme-related information on the storage. Those skilled in the art willappreciate that the Theme Creator/Modifier component can be used tocreate themes and theme-related information in various way, such asthose discussed above or those known in the art for creating ormodifying software components, data structures or visual interfaces. Forexample, the component may provide a Graphical User Interface (“GUI”)with which the user can visually create theme-related data structures,theme layout presentation, or components in an interactive manner. Insome embodiments, the component can present various categories ofcontext information and allow the user to select information from one ormore of the categories (e.g., Me, Location, Activity/Task/Project,People (such as in immediate vicinity), Objects (such as in immediatevicinity), Environment, etc.).

A Theme Distributor component 1440 is also executing in memory todistribute themes and theme-related information to users as appropriate,such as in response to requests or automatically if a user is determinedto meet appropriate criteria. The Theme Distributor component includes aTheme Request Responder sub-component 1442, an Automated ThemeAppropriateness Determiner sub-component 1448, a Theme Sendersub-component 1446, and optionally a Payment Module sub-component 1444.The Theme Request Responder receives requests from users for themesand/or theme-related information, determines if the users are authorizedto receive the requested information, and if so notifies the ThemeSender to provide the requested information. When a theme is beingprovided, the Theme Sender will typically retrieve any othertheme-related information that is associated with the theme, such astheme layouts, theme attributes, theme CSes and/or theme CCs. After thenecessary information is gathered, the Theme Sender will provide theinformation to the user in an appropriate manner (e.g., in an encryptedform).

In some embodiments, the Theme Server may provide themes andtheme-related information as a service to users, such as by charging asubscription service or by charging on a per-theme basis. If there is afee associated with the requested information, the Theme Sender willinteract with the Payment Module to ensure that appropriate compensationhas been received before the information is provided.

The Automated Theme Appropriateness Determiner sub-component alsoprovides themes and theme-related information to users, but does so notin response to user requests but instead to determinations that a usermeets specified criteria (e.g., is a member of a specified group, is ina specified geographic location, is engaged in a specified activity,etc.). In particular, the sub-component receives notifications or othercontext information related to users, and uses the stored user/groupcategorization information and/or theme categorization information todetermine whether any of the themes or other theme-related informationshould be provided to those users. In addition, in some embodiments thesub-component could engage in an automated negotiation with a module ona user computing device to make the appropriateness determination. Whenthe Automated Theme Appropriateness Determiner determines to provideinformation to a user, it notifies the Theme Sender to provide therequested information in a similar manner to that described above. Thoseskilled in the art will appreciate that in some embodiments only one ofthe Theme Request Responder and Automated Theme AppropriatenessDeterminer may be employed as part of the Theme Distributor.

The stored user/group categorization information and themecategorization information that is used by the Automated ThemeAppropriateness Determiner can be generated in a variety of ways, suchas by being received from users or from theme sources. The informationcan also be generated and modified by a user using an Administrationmodule 1434 that is executing in memory. For example, via a GUI providedby the module, a user could specify conditions or executable logic todetermine when to provide themes to users. Those skilled in the art willappreciate that such information can be specified in a variety offormats and used in a variety of ways.

In addition to the illustrated Theme Server computing device, FIG. 14also illustrates a Theme User computing device 1450 suitable forreceiving themes and theme-related information, as well as using theseand other themes in order to provide appropriate responses to the user(not shown) of the Theme User computing device. The Theme User computingdevice and Theme Server computing device communicate via network 1465.

The Theme User computing device includes a CPU 1455, a storage 1470,memory 1480, and various I/O devices 1460, with the storage includingvarious themes and theme-related information in a manner similar to thestorage 1420 of the Theme Server computing device. In particular, thestorage 1470 stores themes 1471, and can also include theme layouts 1472that are each associated with one of the themes, theme-specificattributes 1473 that are part of the set of thematic attributes for atleast one of the themes, theme CSes 1474 able to generate values for oneor more of the theme attributes, theme CCs 1475 able to use informationfrom at least one of the themes to generate appropriate responses forthe user, and theme categorization or grouping information 1477 thatidentifies relationships among the themes.

The memory 1480 similarly includes a group of executing Theme Usagecomponents 1485 that can use the various theme information in a mannersimilar to those of the Thematic Response computing device previouslydescribed. For example, the memory includes an executing Theme Modelercomponent 1487 that repeatedly identifies a current theme set havingdefined themes that match the current context, selects a current theme,and then selects an appropriate type of response (e.g., an associatedtheme layout for presentation or other appropriate actions) based on thecurrent theme. When the Theme Modeler selects an appropriate type ofresponse for a current theme, the Theme Modeler component notifies theexecuting Theme Response Generator component 1488 that provides theappropriate response (e.g., by presenting an appropriate theme layout).Rather than having the Theme Modeler component select the current themein an automated fashion, the user can alternately explicitly specify thecurrent theme using the executing Theme Chooser component 1486 orspecify various context information using the executing Context Choosercomponent 1482. Those skilled in the art will appreciate that executionand use of the various themes can also include the loading and/orexecution of various theme-related information such as theme attributes,theme CSes, and/or theme CCs.

In addition to the Theme Usage components, the memory 1480 also includesvarious executing Theme Administration components 1490. Using thesecomponents, a user can create, modify, categorize, and retrieve themesand theme-related information. In addition, in some embodiments ThemeAdministration components such as the Theme Receiver component 1495 willautomatically receive theme information (e.g., from the Automated ThemeAppropriateness Determiner component 1448 via the Theme Sender component1446) and/or distribute theme information to other devices using a ThemeDistributor component 1497 that operates in a similar manner to ThemeDistributor 1440. The Theme Administration components also include aTheme Creator/Modifier component 1491 with which the user can create ormodify themes and/or theme related information in a similar manner tothat of Theme Creator/Modifier component 1436. For example, the user canuse the component 1491 to explicitly customize themes or theme-relatedinformation received from the Theme Server. The memory also includes aTheme Categorizer component 1492 with which a user can specify themerelationship information (e.g., group, category or hierarchyinformation), such as theme categorization information 1477, for variousthemes (e.g., themes created by the user). In the illustratedembodiment, a user can also search for and generate requests for themesand theme-related information of interest using the Theme Retrievercomponent 1494. Those skilled in the art will appreciate that the ThemeReceiver and Theme Retriever can in some embodiments employfunctionality to provide appropriate access information and/or paymentfor received information. In addition, those skilled in the art willappreciate that in some embodiments only one of the Theme Retriever andTheme Receiver components may be employed as part of the ThemeAdministration modules.

The memory also optionally includes an executing Automated ThemeCustomizer component 1484. In the illustrated embodiment, this componentemploys learning techniques to monitor user actions, detect patterns ofuse related to the various themes, and automatically customize (orsuggest customizations to) received and/or created themes. Those skilledin the art will appreciate that such patterns can be detected in variousmanners (e.g., by using neural networks, expert systems, geneticalgorithms, probabilistic belief networks, etc.), and that a variety oftypes of customizations can occur (e.g., to any of the theme propertiesand/or to theme response information such as theme layouts).

Those skilled in the art will appreciate that computing devices 1400 and1450 are merely illustrative and are not intended to limit the scope ofthe present invention. The computing devices may be connected to otherdevices that are not illustrated, including through one or more networkssuch as the Internet or via the World Wide Web (WWW). In addition, thefunctionality provided by the illustrated components may in someembodiments be combined in fewer components or distributed in additionalcomponents. Similarly, in some embodiments the functionality of some ofthe illustrated components may not be provided and/or other additionalfunctionality may be available. Those skilled in the art will alsoappreciate that, while various components and the context model areillustrated as being stored in memory while being used, these items orportions of them can be transferred between memory and other storagedevices for purposes of memory management and data integrity. Similarly,data illustrated as being present on storage while being used caninstead be present in memory and transferred between storage and memory.Some or all of the components or data structures may also be stored(e.g., as instructions or structured data) on a computer-readablemedium, such as a hard disk, a memory, a network, or a portable articleto be read by an appropriate drive. The components and data structurescan also be transmitted as generated data signals (e.g., as part of acarrier wave) on a variety of computer-readable transmission mediums,including wireless-based and wired/cable-based mediums.

Themes and theme-related information can be distributed in a variety ofways and for a variety of purposes. In some embodiments themes aremodeled using object data structures, with such themes being easilydistributable. In addition, themes can be distributed by theme serversand/or on a peer-to-peer basis by user computing devices that create anduse themes. Themes can also be distributed to various groups, such as byan employer to some or all employees or by a service provider (e.g., ofan entertainment experience) to users of the service. Themes can also bedistributed to users within a specified geographic location, such as bya store owner to customers that enter the store or by a theme server ata tourist location. For example, a retailer can maintain distinct themesfor employees, vendors, contractors, identified or anonymous shoppers. Ashopper's wireless computing system could, upon entering the store,begin a dialog where the user's preference for privacy is determined. Itmay indicate that the shopper is interested in knowing what is on sale,and they are willing to share their personal product preferences, butdecline to identify themselves except by a permanent but arbitrary IDthat is assigned by the store. Their system can then receive and run atheme that, though it may or may not change the UI presentation orinteraction, would deliver relevant data to the system (or to theretailer).

The distribution and use of themes can also support various businesses.For example, some businesses may provide themes and theme-relatedinformation as a service, such as on a subscription of pay-per-themebasis. Themes that provide currently relevant information of varioustypes could be distributed, such as local map information or informationabout nearby businesses (e.g., restaurants or bookstores) of varioustypes. Other businesses can provide services by acting as aclearinghouse or exchange broker for themes, while other businessescould provide search services to locate and provide themes of interest.A user could subscribe to a Theme-Of-The-Month Club (or other timeperiod) in which popular themes are periodically distributed, or abusiness could provide a “bestsellers” list for themes of various types.As noted above, the distribution and use of themes has applications inthe industrial (e.g., for training, to enhance the safety andproductivity of employees, etc.), retail (e.g., for companies tocommunicate with customers, suppliers, and partners, such as to enhancesales and obtain tracking information), and consumer markets (e.g., toprovide enhanced entertainment and productivity tools). In addition,themes can be used on a variety of computing devices, such as wearablecomputers, PDAs, desktop or laptop computers, wireless phones, etc.Themes can also be used to provide new applications, or toenhance/modify the functionality available from existing applications(e.g., a calendar/schedule application).

FIG. 15 is a flow diagram of an embodiment of the Theme Usage routine1500. The routine determines a current theme set and current theme andprovides an appropriate response based on the current theme, as isillustrated and is described in greater detail above.

FIG. 16 is a flow diagram of an embodiment of the Response Generatorsubroutine 1540. The subroutine generates an appropriate response to acurrent theme, as is illustrated and is described in greater detailabove.

FIG. 17 is a flow diagram of an embodiment of the Theme Creator/Modifierroutine 1700. The routine allows a user to create and/or modify themesand theme-related information, as is illustrated and is described ingreater detail above.

FIG. 18 is a flow diagram of an embodiment of the Theme Distributorroutine 1800. The routine distributes themes to users either in responseto requests or as is automatically determined to be appropriate, as isillustrated and is described in greater detail above.

FIG. 19 is a flow diagram of an embodiment of the Theme Receiver routine1900. The routine receives themes from a theme provider either inresponse to a request or when the theme provider sends the theme in anunsolicited manner, as is illustrated and is described in greater detailabove. FIG. 20 is a flow diagram of an embodiment of the Automated ThemeCustomizer routine 2000. The routine determines how to customize themesin an automated manner based on previous user actions and customize thethemes as determined when appropriate, as is illustrated and isdescribed in greater detail above.

A variety of example themes and example uses for themes have beendiscussed. Other examples include the following: a Driving theme thatautomatically tunes the car radio to a favorite news radio station, orto a news station in the morning and a music station in the evening, orcoordinates with context information for car passengers so that aselected radio station is acceptable on everyone; a Fuel Station themethat provides a list of stations that the vehicle can reach (e.g.,prioritized by closest or by cost) and that matches the current contextwith the level of gas in the tank reaches a specified level; a SpeechReminders theme that allows a computing device to act as aself-improvement tool by listening for selected phrases or sounds (e.g.,“umm,” “probly” instead of “probably,” “Feberary” instead of “February,”etc.) or for speech at sound levels below or above thresholds (e.g., todetect shouting inside), and unobtrusively notifies the user inreal-time when detected (e.g., with a beep in an earpiece speaker); aPersonality Improvement theme that could use psychological profileinformation about the user (e.g., from the Meyers-Briggs test) to assistthe user in adapting their personality in a desired way (e.g., ifsomeone is a strong extrovert and has been spending too much time withnon-family activities they could be coached to give more prominence tothe family, or if the user is a timid person the system can coach theuser during a conversation to assert their opinion)—such a theme or arelated theme could even automatically gather personality informationabout the user by monitoring the behavior of the user and suggestingthat the user could improve in certain ways; a Person theme thatprovides information about a specific topic user's affiliation, aprevious encounter with the person including the time/date/location andwhat they discussed, various personal information; etc.; an Object themethat provides information about a specific topic object's owner,location, type/category, tag, etc.; a Place or Location theme thatprovides information about a location's activities/categories,temperature, people currently there, when the user was last there, etc;a Talking About A Topic theme; a Watching TV theme that providesfunctionality to control the remote and displays current or futureshows; a Watching NBA Basketball game that provides information aboutthe teams and provides functionality to order NBA or team paraphernalia;etc.

The above description of the illustrated embodiments is not intended tobe exhaustive or to limit the invention to the precise form disclosed.While specific embodiments of, and examples for, the invention aredescribed for illustrative purposes, various equivalent modificationsare possible within the scope of the invention, as those skilled in therelevant art will recognize. In addition, the various embodimentsdescribed above can be combined to provide further embodiments. All ofthe above mentioned U.S. patents and applications are herebyincorporated by reference. Aspects of the invention can be modified, ifnecessary, to employ the systems, methods and concepts of the variouspatents and applications described above to provide yet furtherembodiments of the invention.

Those skilled in the art will also appreciate that in some embodimentsthe functionality provided by the routines discussed above may beprovided in alternate ways, such as being split among more routines orconsolidated into less routines. Similarly, in some embodimentsillustrated routines may provide more or less functionality than isdescribed, such as when other illustrated routines instead lack orinclude such functionality respectively, or when the amount offunctionality that is provided is altered. Those skilled in the art willalso appreciate that the data structures discussed above may bestructured in different manners, such as by having a single datastructure split into multiple data structures or by having multiple datastructures consolidated into a single data structure. Similarly, in someembodiments illustrated data structures may store more or lessinformation than is described, such as when other illustrated datastructures instead lack or include such information respectively, orwhen the amount or types of information that is stored is altered.

From the foregoing it will be appreciated that, although specificembodiments have been described herein for purposes of illustration,various modifications may be made without deviating from the spirit andscope of the invention. Accordingly, the invention is not limited exceptas by the appended claims. In addition, while certain aspects of theinvention are presented below in certain claim forms, the inventorscontemplate the various aspects of the invention in any available claimform. For example, while only one some aspects of the invention maycurrently be recited as being embodied in a computer-readable medium,other aspects may likewise be so embodied. Accordingly, the inventorsreserve the right to add additional claims after filing the applicationto pursue such additional claim forms for other aspects of theinvention.

1. A method in a computing device for receiving themes representingcontextual situations and customizing the received themes for use inproviding appropriate responses to the represented contextualsituations, the method comprising: receiving at the computing devicemultiple uncustomized themes supplied from one or more external sourcesthat further supply the multiple uncustomized themes to multiple otherdistinct computing devices whose users are distinct from a user of thecomputing device, each of the received themes representing a contextualsituation and associated with a type of response that is appropriate tothe represented contextual situation; customizing at least one of thereceived themes to reflect the computing device and/or the user of thecomputing device; receiving an indication of a current contextualsituation; determining that one of the customized themes represents thecurrent contextual situation; and providing the type of response that isassociated with the one theme.
 2. The method of claim 1 wherein thereceived multiple themes include a first theme from a first externalsource, wherein the first theme is customized by the computing device toreflect the computing device and/or a user of the computing device, andincluding, under control of each of multiple other computing devices:receiving from one of the external sources the first theme; andcustomizing the first theme to reflect the other computing device and/ora user of the other computing device, the customized first theme for theother computing device being distinct from the customized first themefor the computing device.
 3. The method of claim 1 wherein therepresenting of a contextual situation by a theme is with multiplecontext attributes related to that contextual situation and withcriteria for determining values of those context attributes that matchthat contextual situation, wherein the current contextual situationincludes multiple context attribute values, and wherein the determiningthat the one theme represents the current contextual situation isperformed automatically based on the included values of the currentcontextual situation containing values for the context attributes of theone theme that the criteria of the one theme determines to be matchingvalues.
 4. The method of claim 1 wherein the customizing of the receivedthemes is based on explicit indications received from the user of thecomputing device.
 5. The method of claim 1 wherein the customizing ofthe received themes is performed automatically without intervention bythe user.
 6. The method of claim 1 wherein the customizing of thereceived themes is based on likely preferences of the user of thecomputing device that are determined without intervention by the user.7. The method of claim 6 including, after the determining of the likelypreferences and before the customizing of the received theme, receivingexplicit user approval of the customizing.
 8. The method of claim 6including monitoring actions of the user in order to learn the likelyuser preferences.
 9. The method of claim 1 wherein the customizing ofthe received themes is to specialize the received themes to the user ofthe computing device.
 10. The method of claim 1 wherein the customizingof the received themes is to specialize the received themes to thecomputing device.
 11. The method of claim 1 wherein the customizing ofthe received themes is to modify the types of response that areassociated with the received themes.
 12. The method of claim 1 whereinthe customizing of the received themes is to modify the contextualsituations represented by the received themes.
 13. The method of claim 1wherein the receiving of one of the themes includes receiving at leastone context attribute that is specified by that one theme, a contextserver module able to provide values for one or more context attributesspecified by that one theme and/or a context client module capable ofproducing responses appropriate to the contextual situation representedby that one theme, and wherein the customizing of that received themeincludes customizing one of the received context attributes, thereceived context server module, or the received context client module.14. The method of claim 1 wherein the receiving of one of the themesincludes receiving an associated theme layout that specifies informationappropriate to the contextual situation represented by that theme andthat specifies a manner of presenting the specified information, andwherein the customizing of that received theme includes customizing theassociated theme layout.
 15. The method of claim 1 including determiningthat the provided type of response associated with the one theme isinappropriate for the current contextual situation, and automaticallycustomizing the one theme based on the determining.
 16. The method ofclaim 15 wherein the determining is based on a received user indication.17. The method of claim 1 including receiving an indication of a newcontextual situation, determining that no available theme represents thenew contextual situation, and automatically customizing one of theavailable themes based on the determining.
 18. The method of claim 1wherein the provided type of response includes presenting information tothe user of the computing device that is appropriate to the currentcontextual situation.
 19. The method of claim 1 wherein the customizingof one of the received themes is in such a manner that the associatedtype of response for the customized theme includes presentinginformation specific to the user of the computing device.
 20. The methodof claim 1 wherein the contextual situations represented by the receivedthemes are activities of the user of the computing device.
 21. Themethod of claim 1 wherein one of the represented contextual situationsrelates to a situation of the user of the computing device.
 22. Themethod of claim 1 wherein the one represented contextual situationreflects a modeled mental state of the user.
 23. The method of claim 1wherein one of the represented contextual situations relates to asituation of the computing device.
 24. The method of claim 1 wherein atleast one of the represented contextual situations relates to a physicalenvironment.
 25. The method of claim 1 wherein at least one of therepresented contextual situations relates to a cyber-environment of theuser of the computing device.
 26. The method of claim 1 wherein at leastone of the received themes includes permission information thatspecifies how that theme can be customized, and wherein the customizingof that theme is performed in accordance with that permissioninformation.
 27. The method of claim 26 wherein the permissioninformation specifies that the theme cannot be modified, and wherein thecustomizing of that theme in accordance with that permission informationincludes not customizing the theme.
 28. The method of claim 1 whereinthe computing device is a wearable computer.
 29. The method of claim 1wherein the computing device is a cellular phone.
 30. The method ofclaim 1 wherein the computing device is a mobile phone.
 31. The methodof claim 1 wherein the computing device is a handheld device mounted ina holster on a user of the handheld device.
 32. The method of claim 1wherein the computing device is a thin client device.
 33. Acomputer-readable storage medium whose contents cause a computing deviceto receive themes representing contextual situations and customize thereceived themes for use in providing appropriate responses to therepresented contextual situations, by performing a method comprising:receiving multiple themes each representing a contextual situation andassociated with a type of response that is appropriate to therepresented contextual situation, each of the received themesdistributed by a remote source and available to be distributed to one ormore other distinct computing devices; customizing at least one of thereceived themes; determining that one of the received themes representsa current contextual situation; and providing the type of response thatis associated with the one theme.
 34. The computer-readable medium ofclaim 33 wherein the computer-readable medium is a memory of a computersystem.
 35. The computer-readable medium of claim 33 wherein therepresenting of a contextual situation by a theme is with multiplecontext attributes related to that contextual situation and withcriteria for determining values of those context attributes that matchthat contextual situation, wherein the current contextual situationincludes multiple context attribute values, and wherein the determiningthat the one theme represents the current contextual situation isperformed automatically based on the included values of the currentcontextual situation containing values for the context attributes of theone theme that the criteria of the one theme determines to be matchingvalues.
 36. The computer-readable medium of claim 33 wherein thecustomizing of the received themes is performed automatically withoutintervention by a user of the computing device.
 37. Thecomputer-readable medium of claim 33 wherein the customizing of thereceived themes is to specialize the received themes to a user of thecomputing device.
 38. The computer-readable medium of claim 33 whereinthe customizing of the received themes is to specialize the receivedthemes to the computing device.
 39. The computer-readable medium ofclaim 33 wherein the customizing of the received themes is to modify thetypes of response that are associated with the received themes.
 40. Thecomputer-readable medium of claim 33 wherein the customizing of thereceived themes is to modify the contextual situations represented bythe received themes.
 41. The computer-readable medium of claim 33wherein the receiving of one of the themes includes receiving at leastone context attribute that is specified by that one theme, a contextserver module able to provide values for one or more context attributesspecified by that one theme and/or a context client module capable ofproducing responses appropriate to the contextual situation representedby that one theme, and wherein the customizing of that received themeincludes customizing at least one of the received context attributes,the received context server module, and the received context clientmodule.
 42. The computer-readable medium of claim 33 wherein thereceiving of one of the themes includes receiving an associated themelayout that specifies information appropriate to the contextualsituation represented by that theme and that specifies a manner ofpresenting the specified information, and wherein the customizing ofthat received theme includes customizing the associated theme layout.43. The computer-readable medium of claim 33 wherein the method furthercomprises determining that the provided type of response associated withthe one theme is inappropriate for the current contextual situation, andautomatically customizing the one theme based on the determining. 44.The computer-readable medium of claim 33 wherein the method furthercomprises receiving an indication of a new contextual situation,determining that no available theme represents the new contextualsituation, and automatically customizing one of the available themesbased on the determining.
 45. The computer-readable medium of claim 33wherein the provided type of response includes presenting information toa user of the computing device that is appropriate to the currentcontextual situation.
 46. The computer-readable medium of claim 33wherein the contextual situations represented-by the received themes areactivities-of a user of the computing device.
 47. The computer-readablemedium of claim 33 wherein the computing device is a wearable computer.48. The computer-readable medium of claim 33 wherein the computingdevice is a cellular phone.
 49. The computer-readable medium of claim 33wherein the computing device is a mobile phone.
 50. Thecomputer-readable medium of claim 33 wherein the computing device is ahandheld device mounted in a holster on a user of the handheld device.51. The computer-readable medium of claim 33 wherein the computingdevice is a thin client device.
 52. A computing device for receivingthemes representing contextual situations and customizing the receivedthemes for use in providing appropriate responses to the representedcontextual situations, comprising: one or more processors; an inputcomponent configured to receive at least one theme from an externalsource that distributes the at least one theme to multiple computingdevices having distinct users, each of the received themes representinga contextual situation and associated with a type of response that isappropriate to the represented contextual situation; a customizingcomponent that when executed by at least one or more processors isconfigured to customize at least one of the received themes in a mannerspecific to the computing device; and a response provider component thatwhen executed by at least one or more processors is configured todetermine that one of the received themes represents a currentcontextual situation and providing the type of response that isassociated with the one theme.
 53. The computing device of claim 52wherein the customizing component and the response provider componentinclude software instructions stored in a memory.
 54. The computingdevice of claim 52 wherein the computing device is a wearable computer.55. The computing device of claim 52 wherein the representing of acontextual situation by a theme is with multiple context attributesrelated to that contextual situation and with criteria for determiningvalues of those context attributes that match that contextual situation,wherein the current contextual situation includes multiple contextattribute values, and wherein the determining that the one themerepresents the current contextual situation is performed automaticallybased on the included values of the current contextual situationcontaining values for the context attributes of the one theme that thecriteria of the one theme determines to be matching values.
 56. Thecomputing device of claim 53 wherein the customizing of the receivedthemes is performed automatically without intervention by a user of thecomputing device.
 57. The computing device of claim 53 wherein thecustomizing of the received themes is to specialize the received themesto a user of the computing device.
 58. The computing device of claim 53wherein the customizing of the received themes is to specialize thereceived themes to the computing device.
 59. The computing device ofclaim 53 wherein the provided type of response includes presentinginformation to a user of the computing device.
 60. The computing deviceof claim 53 wherein the computing device is a cellular phone.
 61. Thecomputing device of claim 53 wherein the computing device is a mobilephone.
 62. The computing device of claim 53 wherein the computing deviceis a handheld device.
 63. The computing device of claim 53 wherein thecomputing device is a thin client device.
 64. A computer system forreceiving themes representing contextual situations and customizing thereceived themes for use in providing appropriate responses to therepresented contextual situations, comprising: a memory; means forreceiving at least one theme from an external source that distributesthe at least one theme to multiple computing devices having distinctusers, each of the received themes representing a contextual situationand associated with a type of response that is appropriate to therepresented contextual situation; means for customizing at least one ofthe received themes in a manner specific to the computing device; andmeans for determining that one of the received themes represents acurrent contextual situation and providing the type of response that isassociated with the one theme.
 65. The computer system of claim 64wherein the provided type of response includes presenting information toa user of the computer system.
 66. A computer-implemented method fordefining a default theme and customizing that theme for use by aportable computing device in presenting information that is appropriateto a contextual situation represented by the theme, the methodcomprising: defining a default uncustomized theme representing acontextual situation by, receiving a specification of the representedcontextual situation; receiving one or more indications of informationthat is appropriate to the represented contextual situation; anddefining a theme based on the received indications that can be used byany of multiple computing devices to present the appropriate informationwhen a current contextual situation matches the represented contextualsituation; and customizing the defined default theme based on theportable computing device and based on a user of the portable computingdevice so that the portable computing device can use the customizedtheme to present the appropriate information to the user in a mannerspecific to the portable computing device and the user.
 67. The methodof claim 66 including distributing the customized theme to the portablecomputing device.
 68. The method of claim 66 including customizing thedefined default theme for each of multiple other computing deviceshaving distinct users such that each of the customized themes aredistinct.
 69. The method of claim 66 wherein the received specificationof the represented contextual situation includes one or more indicationsof multiple context attributes that are related to the representedcontextual situation and of criteria for determining values of theindicated context attributes that match the represented contextualsituation, and wherein the defined theme is determined to match acontextual situation when context information for that contextualsituation includes values for the indicated context attributes that aredetermined by the indicated criteria to match the represented contextualsituation.
 70. The method of claim 66 wherein the customizing of thedefined theme is based on explicit indications received from a user ofthe portable computing device.
 71. The method of claim 66 wherein thecustomizing of the defined theme is based on likely preferences of auser of the portable computing device that are determined withoutintervention by the user.
 72. The method of claim 71 including, afterthe determining of the likely preferences and before the customizing ofthe received theme, receiving explicit user approval of the customizing.73. The method of claim 66 wherein the customizing of the defined themeincludes specializing the defined theme to the user of the portablecomputing device.
 74. The method of claim 66 wherein the customizing ofthe defined theme includes specializing the defined theme to theportable computing device.
 75. The method of claim 66 wherein thecustomizing of the defined theme is to modify the appropriateinformation for the defined theme.
 76. The method of claim 66 whereinthe customizing of the defined theme is to modify the contextualsituation represented by the defined theme.
 77. The method of claim 66including: receiving an indication of a current contextual situation;determining that a copy of the defined theme that has not beencustomized matches the current contextual situation; and in response,presenting the indicated appropriate information for the defined theme.78. The method of claim 66 wherein the defining of the theme includesassociating a priority with the defined theme, and wherein thecustomizing of the theme includes modifying the priority.
 79. The methodof claim 66 wherein the defining of the theme includes associatingpermission information with the defined theme that specifies types ofaccess to the defined theme that are allowed, and wherein thecustomizing of the theme includes modifying the permission information.80. The method of claim 66 wherein the defining of the theme includesassociating security information with the defined theme, and wherein thecustomizing of the theme includes modifying the security information.81. The method of claim 66 wherein the defining of the theme includesspecifying a relationship of the defined theme to other themes, andwherein the customizing of the theme includes modifying the specifiedrelationship.
 82. The method of claim 81 wherein the specifiedrelationship is a category of themes that is associated with the definedtheme.
 83. The method of claim 66 wherein the defining of the themeincludes associating a theme layout with the defined theme that includesone or more indications of the indicated appropriate information for thedefined theme, and wherein the customizing of the theme includesmodifying the theme layout.
 84. The method of claim 66 wherein thedefining of the theme includes generating a new context server modulecapable of generating context information for the represented contextualsituation or of generating a new context client module capable ofproducing responses appropriate to the represented contextual situation,and wherein the customizing of the theme includes modifying the newcontext server module or new context client module.
 85. The method ofclaim 66 wherein the method is performed by the portable computingdevice.
 86. The method of claim 85 wherein the customizing of the themeis performed automatically without intervention by the user of theportable computing device.
 87. The method of claim 66 wherein the methodis performed by a computing system, and wherein the customizing of thetheme is performed automatically without intervention by a user of thecomputing system.
 88. The method of claim 66 wherein the portablecomputing device is a wearable computer.
 89. The method of claim 66wherein the portable computing device is a cellular phone.
 90. Themethod of claim 66 wherein the portable computing device is a mobilephone.
 91. The method of claim 66 wherein the portable computing deviceis a handheld device mounted in a holster on a user of the handhelddevice.
 92. The method of claim 66 wherein the portable computing deviceis a thin client device.
 93. A computer-readable storage mediumcontaining instructions that when executed cause a computing device todefine a default theme and customize that theme for use in providing aresponse that is appropriate to a contextual situation represented bythe theme, by performing a method comprising: defining a default themerepresenting a contextual situation by, receiving a specification of therepresented contextual situation; receiving one or more indications of atype of response that is appropriate to the represented contextualsituation; and defining a theme based on the received indications thatcan be used by any of multiple computing devices to provide theappropriate type of response when a current contextual situation matchesthe represented contextual situation; and customizing the defineddefault theme in a specified manner so that the customized theme can beused to provide an appropriate type of response in a manner distinctfrom that of the default theme.
 94. The computer-readable medium ofclaim 93 wherein the customizing of the default theme is performed in amanner specific to an indicated computing device and/or to a user of theindicated computing device.
 95. The computer-readable medium of claim 94wherein the method further comprises distributing the customized themeto the indicated computing device.
 96. The computer-readable medium ofclaim 93 wherein the received specification of the representedcontextual situation includes one or more indications of multiplecontext attributes that are related to the represented contextualsituation and of criteria for determining values of the indicatedcontext attributes that match the represented contextual situation, andwherein the defined theme is determined to match a contextual situationwhen context information for that contextual situation includes valuesfor the indicated context attributes that are determined by theindicated criteria to match the represented contextual situation. 97.The computer-readable medium of claim 93 wherein the customizing of thetheme is performed automatically without intervention by any users. 98.The computer-readable medium of claim 93 wherein the method furthercomprises using the customized theme to provide the appropriate type ofresponse when the represented contextual situation occurs.
 99. Thecomputer-readable medium of claim 98 wherein the providing of theappropriate type of response includes presenting information to a userof the computing device.
 100. The computer-readable medium of claim 93wherein the method further comprises using the customized theme toprovide the appropriate type of response to a wearable computer. 101.The computer-readable medium of claim 93 wherein the method furthercomprises using the customized theme to provide the appropriate type ofresponse to a cellular phone.
 102. The computer-readable medium of claim93 wherein the method further comprises using the customized theme toprovide the appropriate type of response to a mobile phone.
 103. Thecomputer-readable medium of claim 93 wherein the method furthercomprises using the customized theme to provide the appropriate type ofresponse to a handheld device.
 104. The computer-readable medium ofclaim 93 wherein the method further comprises using the customized themeto provide the appropriate type of response to a thin client device.105. A computing device for defining a default theme and customizingthat theme for use by a specified computing device in presentinginformation that is appropriate to a contextual situation represented bythe theme, comprising: one or more processors; a theme definitioncomponent that when executed by at least one or more processors isconfigured to define a default theme representing a contextual situationby receiving a specification of the represented contextual situation,receiving one or more indications of information that is appropriate tothe represented contextual situation, and defining a theme based on thereceived indications that can be used by any of multiple computingdevices to present the appropriate information when a current contextualsituation matches the represented contextual situation; and a themecustomization component that when executed by at least one or moreprocessors is configured to customize the defined default theme toreflect a specified computing device so that the specified computingdevice can use the customized theme to present the appropriateinformation in a manner specific to the specified computing device. 106.The computing device of claim 105 wherein the customizing of the defaulttheme to reflect the specified computing device is performed in a mannerbased on a user of the specified computing device.
 107. The computingdevice of claim 105 further comprising a component capable ofdistributing the customized theme to the specified computing device.108. The computing device of claim 105 wherein the receivedspecification of the represented contextual situation includes one ormore indications of multiple context attributes that are related to therepresented contextual situation and of criteria for determining valuesof the indicated context attributes that match the representedcontextual situation, and wherein the defined theme is determined tomatch a contextual situation when context information for thatcontextual situation includes values for the indicated contextattributes that are determined by the indicated criteria to match therepresented contextual situation.
 109. The computing device of claim 105wherein the customizing of the theme is performed automatically withoutintervention by any users.
 110. The computing device of claim 105wherein the computing device is the specified computing device, andfurther comprising a component capable of using the customized theme toprovide the appropriate type of response when the represented contextualsituation occurs.
 111. The computing device of claim 105 wherein thespecified computing device is a wearable computer.
 112. The computingdevice of claim 105 wherein the specified computing device is a cellularphone.
 113. The computing device of claim 105 wherein the specifiedcomputing device is a mobile phone.
 114. The computing device of claim105 wherein the specified computing device is a handheld device. 115.The computing device of claim 105 wherein the specified computing deviceis a thin client device.